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EXECUTIVE  SUMMARY 


a.  The  U.S.  Army  Environmental  Command  (USAEC),  through  the  Environmental 
Quality  Technology  (EQT)  Program,  requested  Aberdeen  Test  Center’s  (ATC)  Military 
Environmental  Technology  Demonstration  Center  (METDC)  to  develop  and  execute  a  plan  to 
determine  and  document,  if  it  exists,  a  level  of  influence  that  operators  may  have  on  unexploded 
ordnance  (UXO)  detection  technology  results.  The  primary  objective  of  the  test  was  to 
determine  this  level  of  influence  and  to  perform  an  analysis  of  operator  detection  activities  to 
identify  the  factors  that  produce  variations  in  operator  performance. 

b.  Until  recently,  the  UXO  and  Countermine  communities  both  have  relied  on  anecdotal 
evidence  to  account  for  the  widely  differing  levels  of  detection  achieved  from  various  operators. 
However,  recent  empirical  investigations  of  operator  influence  in  the  countermine  community 
have  discovered  substantial  variability  in  detection  perfonnance  between  operators  of  both 
currently  fielded  equipment  (AN19/PSS-12)  and  an  advanced  technology  then  under 
development  (HSATMIDS/PSS-14). 

c.  To  date,  there  has  been  no  similar  attempt  to  objectively  define  the  level  of  operator 
influence  in  the  UXO  arena.  This  effort  sought  to  determine  if  similar  individual  differences  in 
operator  performance  exist  and,  if  so,  to  identify  their  bases.  As  in  the  Countermine  work,  this 
effort  also  sought  an  explicit  description  of  the  human  factors  producing  any  differences  found. 
Such  a  description,  which  could  be  cast  as  a  cognitive  model,  holds  potential  to  serve  as  a 
resource  for  designing  operator  training  that  can  maximize  the  potential  of  fielded  UXO 
detection  tools  and  improve  detection. 

d.  ATC  tested  a  total  of  ten  operators  (five  novices  and  five  experts),  using  a  Schonstedt 
magnetometer.  The  experts  had  more  experience  with  geophysical  detection  than  the  novices. 

e.  The  testing  indicated  anomalies  in  some  of  the  results  relating  to  expert  versus  novice 
performance.  The  overall  performance  of  the  novices  was  better  than  the  perfonnance  of  the 
experts.  The  variability  of  the  novices’  probability  of  detection  (Pd)  results  was  less  affected  by 
factors  such  as  detector  head  height  and  velocity  than  the  variability  of  the  experts’  results.  In 
addition,  Pd  was  affected  diametrically  by  detector  head  height  for  the  novices  versus  the  experts. 

f.  Overall,  the  results  showed  that  that  the  position  and  speed  of  the  detector  head 
impacted  perfonnance  measurements.  In  addition,  the  data  indicated  that  perhaps  periodic 
refresher  training  would  be  beneficial  to  expert  operators  to  improve  their  results  in  the  field. 

h.  This  effort  provides  but  a  glimpse  into  operator  influence  of  UXO  detection  and  bears 
further  study.  Increasing  overall  sample  size  would  allow  cogent  conformation  of  this  initial 
study  and  an  inferential  statistical  analysis  versus  the  descriptive  analysis  accomplished  herein. 
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1.1  TEST  OBJECTIVES 


a.  The  primary  objective  of  this  test  program  was  to  quantitatively  detennine  the  level  of 
influence  that  individual  operators  have  on  unexploded  ordnance  (UXO)  sensors.  This  level  of 
influence  was  first  evaluated  by  observing  the  perfonnance  of  expert  and  novice  operators 
engaged  in  a  blind  search  for  a  variety  of  UXO  targets.  Empirical  data  were  collected  for  both 
novice  and  expert  perfonnance  levels  with  a  commercial  handheld  magnetometer.  Data  recorded 
were  scored  on  traditional  metrics  such  as  probability  of  detection  (Pd),  false  alarm  rates  (FAR), 
and  time  during  the  course  of  traversing  a  one-third  acre  test  plot.  This  information  was  intended 
to  provide  insight  into  maximizing  operator  performance. 

b.  In  summary,  the  main  focus  of  this  effort  was  to  observe,  identify,  and  describe 
operator  actions  during  UXO  detection  operations. 

1.2  TEST  AUTHORITY 

The  U.S.  Army  Aberdeen  Test  Center  (ATC)  Military  Environmental  Technology 
Demonstration  Center  (METDC)  performed  testing  at  Aberdeen  Proving  Ground  (APG), 
Maryland.  The  APG  Standardized  UXO  Technology  Demonstration  Site  was  used  for  the  test. 
The  U.S.  Anny  Environmental  Command  (USAEC)  under  the  Army  Environmental  Quality 
Technology  (EQT)  Program  sponsored  the  test.  This  test  was  performed  under  U.S.  Army 
Developmental  Test  Command  (DTC)  Project  No.  8-CO-160-UXO-023  in  support  of  the  user 
requirement  outlined  in  the  EQT  A  (1.6a)  UXO  Screening,  Detection  and  Management  Plan. 
The  U.S.  Anny  Research  Laboratory  (ARL)  Human  Factors  group  and  Concurrent  Technologies 
Corporation,  Inc.  (CTC)  provided  support  to  assess  the  human  factors  directly  relevant  to  the 
objectives  and  methods  described  herein. 

1.3  TEST  CONCEPT 

a.  Highly  experienced  users  of  the  Schonstedt  device  (experts)  were  tested  in  UXO 
detection  as  well  as  operators  who  recently  completed  training  on  the  instrument  (novices).  Ten 
test  participants,  comprising  five  experts  and  five  novices  in  the  field,  were  chosen  to  participate 
in  the  testing. 

b.  The  novice  participants  were  civilian  trained  UXO  technicians.  These  individuals  were 
certified  as  Level  I  UXO  Technicians  through  training  received  through  the  Texas  Engineering 
Extension  Service  (TEEX),  an  extension  of  the  Texas  Agricultural  and  Mechanical  (A&M) 
University  System  (TAMUS).  The  5-week  training  curriculum  emphasizes  environmental 
theory,  ordnance  identification,  safety,  and  explosives  with  practical  experience  time  allotted  for 
detection  equipment. 

c.  The  expert  participant  was  required  to  have  prior  military  experience,  successful 
completion  of  an  Explosive  Ordnance  Disposal  (EOD)  Program,  and  previous  UXO  detection 
experience.  Expert  participants  were  randomly  selected  from  a  Government  organization 
currently  engaged  in  active  UXO  site  remediation.  Personnel  were  fluent  with  current  Tactics, 
Techniques  and  Procedures  (TTPs)  as  well  as  the  specific  instruments  used  in  this  study. 
Overall,  the  expert  participant  was  required  to  demonstrate  a  high  level  of  confidence  with  the 
detection  technology. 
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d.  Inclusion  of  expert  operators  in  the  test  introduced  operator  skill/experience  as  a 
dimension  of  operator  differences  likely  to  account  for  variability  in  performance  hypothesized 
on  the  basis  of  the  referenced  countennine  findings.  The  operators  were  instrumented  to  capture 
qualitative  information  capable  of  revealing  how  they  used  their  equipment  and  the  information 
it  provided  to  detect  targets.  It  was  hypothesized  that  experts  would  produce  the  highest 
detection  performance. 

e.  To  date,  no  similar  attempt  has  been  made  to  objectively  define  the  level  of  operator 
influence  in  the  UXO  arena.  This  effort  was  executed  to  determine  if  similar  individual 
differences  in  operator  performance  exist  and,  if  so,  to  identify  their  sources.  Similar  to  the 
countermine  work,  this  effort  provides  an  explicit  description  of  the  human  factors  producing 
any  differences  found.  It  is  the  intent  of  the  Test  and  Evaluation  team  that  data  observed  could 
be  a  resource  for  designing  operator  training  that  can  maximize  the  potential  of  fielded  UXO 
detection  tools  and  improve  detection  capability. 

f.  The  field  portion  of  the  test  commenced  during  September  2004  and  continued  through 
November  2005.  The  test  participants  were  monitored  using  several  methods  throughout  the 
practical  exercise,  and  the  results  were  recorded,  processed,  evaluated,  and  scored.  Several 
human  factors  were  evaluated,  including  (but  not  limited  to)  physiological  stress/anxiety  levels, 
height  of  the  operator  instrument  with  reference  to  ground,  sweep  rate,  and  walking/pacing  or 
velocity  of  the  operator.  The  final  product  will  summarize  results  of  observed  differences  in 
trends  of  expert  versus  novice  UXO  technicians.  A  flow  chart  of  the  overall  test  concept  is 
provided  in  appendix  C. 

g.  An  integrated  team  was  established  for  the  review,  comment,  direction,  and  conduct  of 
this.  Members  of  this  team  included: 

(1)  ATC,  DTC. 

(2)  ARL  Human  Research  and  Evaluation  Directorate. 

(3)  U.S.  Anny  Corps  of  Engineers  (ACE)  -  Huntsville. 

(4)  Concurrent  Technologies  Corporation  (CTC). 

(5)  Scientific  Research  Corporation  (SRC). 
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1.4  SYSTEM  DESCRIPTION 


a.  For  this  investigation,  participants  used  a  magnetometer  commonly  found  within  the 
UXO  community. 

b.  The  Schonstedt  model  52-CX  (fig.  1-1)  is  a  magnetometer  designed  to  locate  changes 
in  magnetic  energy  fields.  Intended  originally  for  locating  subsurface  ferrous  infrastructure 
components,  this  device  was  also  found  to  be  effective  at  locating  buried  UXO. 


Figure  1-1.  Schonstedt  magnetic  locator. 


c.  The  52-CX  uses  two  coaxially  mounted  magnetic  sensors  within  its  nonmagnetic 
structure.  The  output  signals  of  the  two  sensors  are  directed  such  that  they  oppose  each  other. 
When  the  axis  of  the  sensors  is  located  within  a  unifonn  magnetic  field,  the  components  of  that 
magnetic  field  are  equal  and  opposite.  Thus,  no  signal  is  outputted  through  an  onboard  speaker 
within  the  nonmagnetic  structure  of  the  52-CX.  If  the  axis  of  the  sensors  does  encounter  a 
magnetic  field  other  than  that  of  the  earth’s  natural  unifonn  field,  the  rates  of  magnetic  flux  and 
overall  field  will  generally  be  higher  at  one  sensor.  Therefore,  a  net  difference  between  the  two 
sensors  will  be  observed  and  an  audible  output  will  be  provided  to  the  operator.  This  audible 
output  will  change  frequency  and  intensity  as  the  net  magnetic  field  changes.  The  52-CX  has  a 
five-step  potentiometer  that  allows  the  user  to  account  for  high  levels  of  subsurface  background 
metallic  content  by  changing  sensitivities  of  the  magnetic  field  sensors.  In  addition,  this  allows 
the  user  to  customize  the  operation  of  the  detector  for  specific  target  sizes.  For  instance,  larger 
targets  will  often  saturate  the  localized  magnetic  field.  An  operator  will  change  the  sensitivity  of 
the  52-CX  to  account  for  this  phenomena  so  as  to  allow  for  smaller  targets  to  be  identified  and 
located.  The  detector  is  powered  by  two  9-volt  alkaline  batteries.  Its  simple  operation  and  ease 
of  use  account  for  its  popularity  within  the  UXO  community. 

d.  No  modifications  were  made  to  the  Schonstedt  for  testing  purposes. 
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e.  Operators  were  required  to  use  headphones  while  operating  the  Schonstedt  Model 
52-CX.  A  variable  potentiometer  was  used  to  control  the  volume  output  from  the  external 
speaker.  This  field-hardened  potentiometer  was  constructed  by  ARL  and  attached  to  the 
Schonstedt  headphones-jack  output  by  the  existing  external  audio  connection.  This  functioned 
as  a  variable  resistor  and  limited  current  flowing  into  connected  headphones,  based  on  the  radial 
position  of  the  knob.  The  audio  signal  was  delivered  to  the  operator  via  Audio  Technical  Model 
ATH-M30  headphones. 

1.5  UNIQUE  TEST  REQUIREMENTS 

a.  Test  Equipment. 

(1)  A  tracking  system  capable  of  observing  and  recording  dynamic  motions  of  UXO 
detection  systems  was  required  for  this  test.  The  system  was  essential  to  input  Universal 
Transverse  Mercator  (UTM)  test  grid  boundary  as  well  as  target  location  coordinates. 

(2)  A  potentiometer  to  control  audio  output  of  all  UXO  detection  systems  was  required 
for  this  test. 

(3)  The  participants  were  provided  with  all  equipment  necessary  for  testing 

(4)  Audio/visual  equipment  capable  of  documenting  static  and  dynamic  test  operations. 

b.  Personnel.  Two  on-site  observers  monitored  the  field  operations.  Test  staff  recorded 
data  and  signal  processing  activities,  operated  data  collection  equipment,  and  conducted 
maintenance  activities.  Geodetics  support  was  required  to  operate  telemetry  equipment  and 
survey  locations  of  operator  declarations. 

c.  Field  Activities. 

(1)  Target  emplacement. 

(a)  The  objective  during  this  phase  of  testing  was  to  emplace  the  UXO  test  items  (targets) 
within  the  area  designated  as  the  test  bed. 

(b)  Targets  were  selected  from  the  Standardized  Site  Repository  at  APG.  The  targets 
represent  five  ordnance  types,  including  40mm  projectiles,  60mm  mortars,  81mm  mortars, 
105mm  projectiles,  and  155mm  projectiles.  These  ordnance  types  were  chosen  because  they  are 
indicative  of  the  common  munitions  found,  readily  available,  and  representative  of  various 
associated  aspect  ratios  and  sizes.  Sixty  targets  were  emplaced.  Sketches  of  the  ordnance  are 
provided  in  appendix  H. 

(c)  Prior  to  target  emplacement,  all  items  were  degaussed  in  accordance  with 
MIL-M- 19595  by  ATC  personnel.  The  process  of  degaussing  or  demagnetization  ensured  that 
any  magnetic  flux  stored  in  the  munitions  was  near  or  close  to  zero  upon  entrenchment.  The 
munitions  were  then  separated  in  crated  compartments,  stored,  and  secured  to  maintain  the 
integrity  of  the  magnetic  field. 
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(2)  Test  Procedures. 

(a)  The  test  bed  was  located  within  the  APG  Standardized  UXO  Technology 
Demonstration  Test  Site  (fig.  1-2)  between  the  calibration  test  area  and  the  blind  grid  test  area. 
The  location  selected  for  the  test  bed  is  approximately  one-third  acre  in  size.  Grid  lane  spacing 
was  fixed  at  1.5-meter  widths  in  accordance  with  standard  practices.  Geodetics  support  was 
necessary  to  verify  the  coordinates  for  test  bed  boundaries  and  target  emplacement/ground  truth. 


Figure  1-2.  APG  Standardized  UXO  Technology  Demonstration  Site. 


(b)  Sixty  ordnance  items  were  emplaced  within  the  test  bed.  No  clutter  was  emplaced.  It 
was  intended  that  no  signatures  from  neighboring  items  would  overlap;  therefore,  the  halo  size 
was  selected  to  accommodate  the  largest  item,  a  155mm  projectile.  The  default  halo  for  this  item 
was  chosen  to  be  1  meter  in  diameter.  Orientations  of  0  and  90°  magnetic  north  were  chosen 
because  of  the  local  maximum  and  minimum  amplitudes  of  the  magnetic  fields  at  these  angles. 

(c)  Detection  of  the  60  ordnance  items  with  a  two-failure  allotment  resulted  in  a 
90-percent  reliability  with  a  95-percent  confidence  rating.  After  emplacement  of  the  targets,  the 
area  was  reseeded  and  maintained  to  minimize  visual  and  physical  evidence  of  target  locations. 
Additional  soil  and  seeding  was  required  to  compensate  for  settling  effects.  The  site  was  also 
allowed  to  age  and  weather  for  8  months  before  testing  to  further  minimize  physical  and  visual 
evidence  of  target  locations.  Any  soil  that  was  used  as  fill  had  the  same  composition  and 
properties  of  the  soil  already  found  on  the  site,  as  stated  in  the  Standardized  UXO  Technology 
Demonstration  Site  Handbook.  The  area  was  periodically  inspected  for  signs  of  erosion  and/or 
target  exposure. 

(d)  All  targets  emplaced  within  the  test  bed  were  within  the  95th  percentile  of  the 
maximum  recovered  depth  listed  in  the  ACE  Recovery  Depths  Database.  Each  ordnance  type 
was  buried  at  an  assigned  target  depth.  The  target  types  and  corresponding  depths  are  provided 
in  table  A-l  in  appendix  A. 
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1.6  TESTING  METHODOLOGY 


The  objective  of  this  phase  of  testing  was  to  observe  and  record  UXO  Technicians  while 
executing  a  MAG  and  flag  operation  on  a  pre-seeded  test  grid. 

a.  Requirements. 

(1)  Novice  operators  completed  the  5-week  course  for  certification  as  a  UXO  Technician 
Level  I  at  Texas  A&M  University  System,  TEEX  (app  B). 

(2)  Training  variations  were  expected  for  each  expert  operator.  At  a  minimum,  each 
expert  operator  was  required  to  have  5  years  of  experience  in  the  UXO  detection  field  and 
successful  completion  of  a  Department  of  Defense  (DOD)  certified  training  program.  Operators 
with  previous  military  experience  were  required  to  have  certification  from  the  Anny  Bomb 
Disposal  School  located  at  APG  or  the  Naval  Explosive  Ordnance  Disposal  (EOD)  School, 
Indian  Head,  Maryland. 

b.  Test  Procedures. 

(1)  Five  participants  of  each  group  (novice/expert)  used  an  identical  Schonstedt  model 
52-Cx  instrument  to  survey  the  designated  test  bed.  The  Schonstedts  were  checked  for 
serviceability  between  operators.  Each  operator  was  provided  identical  information  about  the 
objectives  of  the  test  and  instructions.  The  operators  underwent  a  hearing  assessment  performed 
by  a  certified  audiologist  prior  to  the  demonstration. 

(2)  A  practice  session  was  conducted  to  familiarize  each  operator  with  the  task.  Each 
operator  was  allotted  time  in  a  test  pit.  This  area  consisted  of  nine  targets  located  on  surface 
covered  with  inverted  plastic  buckets.  Each  bucket  was  cut  to  a  recorded  height  that  allowed 
instruments  to  respond  to  the  various  magnetic  fields  of  each  target.  The  40mm  grenade 
projectiles  were  oriented  in  a  horizontal  and  vertical  position.  Three  heights  of  each  orientation 
allowed  operators  to  experience  the  full  range  of  magnetic  signatures.  Ten-pound  shot  puts  were 
placed  under  three  plastic  buckets  also  at  varying  heights.  The  shot  puts  and  40mm  grenade 
provided  both  ferrous  and  nonferrous  situations  for  the  operator  to  encounter  prior  to  testing. 
Photographs  of  the  test  pit  are  provided  in  appendix  G.  The  duration  of  each  test  bed  period  was 
recorded  in  the  on-site  daily  logbook.  Only  one  operator  was  allowed  in  the  test  bed  during  test 
activities. 

(3)  The  survey  portion  of  the  test  was  expected  to  take  approximately  6  hours  based  on 
prior  knowledge  from  subject  matter  experts  at  the  operational  level  of  UXO  remediation.  Each 
participant  was  pennitted  time  for  standard  field  operations,  such  as  mobilization/setup, 
calibration,  and  demobilization  as  well  as  time  to  address  any  equipment  issues  that  surfaced 
during  testing. 

(4)  Throughout  the  survey  process,  test  staff  strived  to  adhere  to  consistency  in 
environmental  factors  relating  to  weather  and  conditions  on  the  field.  The  start  and  stop  times  of 
each  operator  were  recorded  to  compute  survey  completion  times. 
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(5)  Each  participant  was  instructed  that  they  would  be  traversing  the  test  plot  in  two 
instances:  with  and  without  the  laser  tracking  system.  Thus,  each  participant  scanned  the  same 
test  grid  twice,  with  the  same  UXO  detection  equipment.  Test  participants  were  directed  to 
begin  on  opposite  sides  of  the  test  grid  between  iterations  to  limit  memory  from  previous  sweeps. 
In  addition,  one  sweep  was  limited  per  day;  therefore,  no  person  could  complete  both  sweeps  in 
the  same  day. 

(6)  The  stress  data  were  collected  during  both  iterations  for  each  individual.  A  test  staff 
member  administered  the  subjective  questionnaires  and  saliva  samples  during  the  test. 

(7)  The  audiological  testing  was  performed  only  once  at  the  beginning  of  the  test. 

c.  Data  Required.  The  data  required  are  provided  in  appendix  F.  The  required  test 
observational  data  included  all  recorded,  signal  processing,  and  operational  field  data;  video 
footage;  auditory  testing  results;  detector  head  height;  and  sweep  rate.  Start  and  stop  times  of  the 
operators,  as  well  as  time  spent  performing  calibration  and  mobilization,  were  recorded  on-site. 
The  operators  were  required  to  have  a  hearing  assessment  prior  to  the  test. 

d.  Objective.  This  phase  of  testing  will  describe  the  methodology  for  analyzing  observed 
and  recorded  data  during  UXO  sweep  operations. 

e.  Requirements. 

(1)  Two  main  stages  of  data  handling  were  performed  during  testing.  The  first  stage 
concentrated  on  acquisition  and  represented  gathering  and  capturing  information  of  each 
participant  as  testing  was  completed.  Data  captured  were  reported  on  an  operator-by-operator 
basis.  The  second  stage  of  data  analysis  included  identification  of  any  trends  between  common 
sets  or  groupings  within  the  testing  parameters. 

(2)  ATC  Statistical  and  CTC  analyzed  the  data  to  obtain  the  metrics  for  scoring  Pd,  and 
false  alarm  rate  (FAR)  for  each  novice  and  expert  operator.  The  submittal  reflected  response 
stage  (Pdres)  scoring  only.  The  perfonnance  rating  data  were  further  reviewed  for  any 
correlations  with  the  test  observational  data. 

(3)  The  captured  data  and  descriptions  are  presented  in  Table  1-1.  Further  details  are 
provided  in  appendix  E. 
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TABLE  1  - 1 .  LIST  OF  DATA 


Detection 

Results 

Operator 

Performance 

Ordnance 

Stress 

Hearing 

Demographics 

Pd 

Forward  velocity 

Type 

Anxiety 

PTA 

Age 

BAR 

Detector  height 

Depth 

Depression 

Gender 

Sweep  rate 

Orientation 

Hostility 

Education  | 

Total  time 

Azimuth 

Positive 

Affect 

uxo 

experience 

Average  lane  time 

Dysphoria 

Detector 

experience 

%  area  covered 

Salivary, 

amalase 

Workload 

Cortisol 

(4)  Analysis  of  test  observational,  cognitive,  and  relational  data  constituted  the  second 
stage  of  analysis,  which  was  perfonned  by  CTC,  ARL,  and  ATC.  The  data  were  analyzed  and 
the  results  of  the  test  were  documented  to  detennine  if  qualitative  factors  and  differences  in 
performance  impacted  operator  scores. 

f.  Data  Acquisition. 

(1)  The  Threat  Minefield  System  (TMS)  was  used  to  capture  real-time  motions  of 
detector  shaft  and  the  operator.  Originally  intended  and  designed  for  the  countermine 
community,  this  device  was  constructed  for  virtual  mine  operations. 

(2)  TMS  consisted  of  a  laser-based  tracking  technology.  Optical  receivers  were  attached 
to  the  shaft  of  the  detection  equipment  and  the  operator’s  feet.  Four  rotating  lasers  were 
positioned  around  the  perimeter  of  the  test  course.  The  rotating  lasers  provided  a  virtual 
3 -dimensional  volume  of  laser  energy.  This  laser  energy  was  received  by  optical  sensors 
positioned  on  the  operator’s  feet  and  detection  equipment  shaft.  An  onboard  processor  then 
calculated  the  position  of  the  sensors  and  reported  absolute  positioning  via  wireless  link  to  a 
master  computer  control  system.  Telemetry  data  were  stored  onboard  the  master  controller 
computer  in  a  dedicated  hard  drive. 

(3)  Video  documentation  of  testing  consisted  of  digital  video,  streaming  video,  and  static 
photographs.  A  live  real-time  video  feed  of  the  detector-head  was  provided  and  sent  via  TMS 
wireless  link  to  the  master  control  computer.  This  was  saved  with  any  operator  tracking  data. 
Digital  video  of  an  operator’s  lane  coverage  was  obtained  during  the  test.  The  video  was 
captured  at  the  opposing  end  of  a  lane,  which  recorded  the  sweeping  portion  and  end  of  that  lane. 
A  post  debrief  was  conducted  interviewing  each  operator  to  obtain  comments  or  suggestions. 
This  video  was  edited  and  consolidated  for  ancillary  data  input  to  analysis.  Static  still 
photographs  were  acquired  throughout  testing  for  documentation  purposes.  Every  effort  was 
made  to  record  common  situations  encountered  during  testing  procedures  (e.g.,  lane  coverage, 
bucket  test,  equipment  layout,  field  positioning). 


(4)  A  daily  logbook  was  kept  with  time  data. 
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(5)  The  factor  of  human  stress  was  captured  and  quantified  by  subjecting  UXO 
technicians  to  a  written  battery  of  questions  as  well  as  incorporating  saliva  samples 
measurements  at  various  times  throughout  the  test.  The  workload  was  also  measured  via  written 
questionnaire.  The  ARL’s  Human  Research  and  Engineering  Directorate  served  as  subject 
matter  experts  on  this  portion  of  the  testing. 

g.  Data  Analysis/Procedures. 

(1)  Detection  results  were  organized  and  data  reduction  was  performed.  This  process 
confirmed  that  usable  information  was  recorded  and  showed  where  any  data  gaps  existed  from 
test  dates.  Data  gaps  were  noted  and  recorded.  Metrics  were  organized  by  individual  operator, 
date,  and  grouping  of  operator  (expert  or  novice). 

(2)  Data  were  compartmentalized  and  separated  into  subgroups  that  consisted  of  the 
following: 

(a)  Manual  information  was  gathered:  data  generated  by  recording  target  declarations 
manual  with  GPS. 

(b)  Telemetry  information  was  gathered:  data  observed  and  recorded  with  the  TMS 
system  including  target  declarations,  movement  patterns,  detector  height,  sweep  rate,  lane  times, 
and  forward  velocity. 

(c)  Video  record  was  made:  real-time  captured  video  from  TMS  system,  operator  lane 
recordings,  and  static  digital  photography. 

(d)  Audiology  testing  was  performed:  hearing  test  results  from  each  operator  and 
detection  equipment  audio-output  sound  characterization. 

(e)  Psychological  evaluation  was  perfonned:  stress  test  results  from  both  questionnaires 
and  salivary  amylase  sampling.  These  results  confirmed  operator  stress  levels  at  both  a 
qualitative  and  quantitative  sense. 

(f)  On-site  observations  were  made:  general  observations  from  test  personnel  located  on¬ 
site  during  test  operations. 

(3)  A  priority  of  review  was  given  to  the  fundamental  performance  data.  Elements  such 
as  Pd,  FAR,  time,  and  detector  height  are  the  primary  indicators  of  operator  performance.  Pd  and 
FAR  are  two  traditional  elements  reviewed  to  determine  the  performance  of  emerging  UXO 
detection  technologies.  Time  is  a  critical  data  point,  as  it  is  directly  proportional  to  the  cost 
associated  with  typical  UXO  clearance  operations.  Detector  height  can  be  critical,  as  it  may 
relate  to  operator  performance  and  effectiveness  in  locating  buried  items.  It  is  currently 
unknown  what,  if  any,  critical  height  must  be  maintained  to  provide  the  greatest  probability  of 
locating  UXO  with  the  least  amount  of  background  alarms. 

(4)  Multiple  target  declarations  within  1 -meter  halos  were  consolidated,  and  corrected 
values  were  compared  with  ground  truth.  This  allowed  for  artificially  inflated  FAR  score  due  to 
multiple  target  declaration. 
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(5)  Analyses  of  perfonnance  examined  the  effects  of  individual  operators,  operator 
experience  level,  equipment,  target,  and  types.  Procedures  involved  aggregation  of  data  by 
conditions,  examination  of  resulting  distributions,  and  selection  of  appropriate  inferential 
statistical  tests. 

(6)  Trends  produced  by  data  were  recorded.  Sources  for  these  trends  were  investigated 
by  reviewing  all  perfonnance  information  on  file. 

(7)  An  enor  analysis  examined  the  qualities  of  any  targets  missed  and  related  the  qualities 
of  the  missed  targets  to  operator  variables,  individual  operator’s  performance  indexes,  and 
equipment  used. 

h.  Organization  and  Responsibilities. 

(1)  USAEC  provided  overall  management  and  funding  of  the  test  program. 

(2)  ATC  was  the  lead  agency  for  preparation  of  the  DTP  and  test  report;  operation  of  the 
test  facility;  conduct  of  the  test;  real-time,  laser-based  human  tracking  system  operation;  still 
photographic  documentation;  real-time  kinematics  (RTK)  global  positioning  system  recording  of 
declared  targets;  statistical  regression  modeling  manually  marked  GPS  targets;  and 
communicating  with  other  members  of  the  team. 

(3)  ARL  Human  Research  and  Engineering  Directorate  perfonned  auditory  testing  of  test 
participants;  sound  modeling  and  characterization  of  UXO  detection  equipment;  odification  of 
any  UXO  equipment  for  proper  audio  output;  stress  analysis  of  all  test  participants;  and  analysis 
of  data  for  input  into  test  report. 

(4)  CTC  performed  analysis  of  telemetry  data  and  general  input  into  test  report  under 
contract  to  ATC. 

(5)  SRC  performed  upgrades  to  existing  TMS  hardware  and  software  system  and  general 
information  technology  related  support  under  contract  to  ATC. 
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SECTION  2.  UN-INSTRUMENTED  DATA 


2.1  EXPERT 

2.1.1  Performance  Measurements 


a.  Probability  of  Detection  (Pd). 

(1)  For  each  alarm  an  operator  noted  in  the  lane,  the  distance  between  each  of  the  targets 
in  the  field  and  the  alarm  was  calculated.  If  the  distance  was  less  than  1  meter,  then  a  target  was 
considered  detected  regardless  of  which  lane  the  target  was  in.  Multiple  detections  of  the  same 
target  were  ignored.  The  number  of  detected  targets  in  the  field  were  divided  by  the  total 
number  of  targets  in  the  field  (60  targets)  and  multiplied  by  100.  This  result  was  defined  as  the 
Pd  and  will  be  referred  to  as  Pd.  Pd  is  a  dimensionless  number  with  values  ranging  from  zero  to 
one. 


(2)  Expert  participants  demonstrated  Pd  rates  from  0.917  to  1.  Their  average  Pd  was 
0.957  with  a  standard  deviation  of  0.0401. 

b.  False  Alarm  Rate  (FAR). 

(1)  For  each  alarm  an  operator  declared  in  a  particular  lane,  the  distance  between  each  of 
the  targets  in  the  field  and  the  alarm  was  calculated.  If  no  target  was  within  1  meter  of  the  alarm, 
then  the  alarm  was  considered  a  false  alann.  The  total  numbers  of  false  alarms  were  divided  by 
the  area  of  the  field  (1131.5  square  meters).  The  result  was  defined  as  the  FAR  and  will  be 
referred  to  as  FAR.  FAR  has  a  unit  of  false  alarms  per  meters  squared. 

(2)  Expert  participants  demonstrated  FAR  rates  from  0.034  to  0.154  false  alanns  per 
meters  squared.  The  average  FAR  was  0.087  per  meters  squared  with  a  standard  deviation  of 
0.0435  false  alarms  per  meters  squared. 

c.  Distance  from  Optimal  Point  (DOP). 

(1)  An  ROC  curve  is  an  industry  standard  that  is  used  to  compare  the  performance  of 
operators  and  equipment  in  UXO  and  mine  detection.  It  consists  of  the  FAR  on  the  x-axis  versus 
the  Pd  on  the  y-axis.  Curves  nearer  the  upper  left-hand  comer  of  the  chart  are  considered  to  be 
higher  performance  from  a  detector.  Therefore,  in  order  to  compare  the  operators’  perfonnance 
versus  each  of  the  characteristics,  the  DOP  was  calculated  as  the  distance  from  the  upper 
left-hand  corner  (coordinates  0,  1)  that  an  operators’  point  (FAR,  Pd)  is  on  the  ROC  curve,  as 
shown  in  the  following  equation: 


Dist  _  DOP  =  J(l-Pd)2  +  (0-FAR)2 


2.1-1 


(2)  The  DOP  value  is  a  dimensionless  number;  however,  in  general  the  greater  the  value 
the  lower  overall  performance  within  Pd  and  FAR  dimensions.  Expert  participants  demonstrated 
DOP’s  between  0.075  and  0.154.  Their  average  DOP  was  0.108  with  a  standard  deviation  of 
0.0302. 

2.1.2  Time  Measurements 


a.  Lane  Velocity. 

(1)  The  time  operators  required  to  complete  each  lane  was  manually  recorded  in  an 
on-site  daily  log.  Time  delays  due  to  equipment  issues  or  data  recording  were  also  recorded  and 
then  subtracted  from  the  total  lane  time.  This  lane  length  was  then  divided  by  the  “corrected” 
lane  time.  The  result  was  defined  as  the  lane  velocity. 

(2)  Expert  participants  demonstrated  lane  velocities  between  0.088  m/s  and  0.01  m/s. 
The  average  lane  velocity  was  0.094  m/s  with  a  standard  deviation  of  0.0044. 

b.  Total  Time. 

(1)  The  time  operators  required  to  execute  a  UXO  sweep  operation  on  all  33  lanes  of  the 
test  site.  Corrected  lane  times  were  used  for  the  following  summation: 


33 


Total  _  Time  =  J'j  _  lanes 


i= 1 


(2)  Expert  participants  demonstrated  total  times  between  8199  and  9086  seconds.  The 
average  total  times  were  8482  seconds  with  a  standard  deviation  of  374.7  seconds. 

2.1.3  Data  Summary 


TABLE  2.1-1.  EXPERT  PERFORMANCE  DATA 


1 

Operator 

P„ 

FAR, 

1/m2 

DOP 

Lane 

Velocity, 

m/s 

Total 
Time,  s 

Experts 

E-l 

100.0% 

0.154 

0.154 

0.095 

8238 

E-2 

91.7% 

0.072 

0.110 

0.088 

9086 

E-3 

100.0% 

0.087 

0.087 

0.100 

8279 

E-4 

93.3% 

0.034 

0.075 

0.092 

8608 

E-5 

93.3% 

0.090 

0.112 

0.096 

8199 

Mean 

95.7% 

0.087 

0.108 

0.094 

8482 

Std.  Dev. 

0.0401 

0.0435 

0.0302 

0.0044 

374.7 
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2.1.4  Ordnance.  Five  different  ordnance  types  were  used:  40mm,  60mm,  81mm,  105mm,  and 
155mm.  Each  type  was  buried  at  a  specific  depth:  6,  12,  18,  24,  and  30  inches.  The  ordnance 
was  placed  in  both  the  horizontal  and  vertical  orientations.  The  Pd  measurements  for  each 
ordnance  type,  depth,  and  orientation  are  presented  in  Table  2.1-2. 


TABLE  2.1-2.  EXPERTS  WITHOUT  INSTRUMENTATION  -  Pd  BY  ORDNANCE 

ORIENTATION  AND  TYPE 


Type 

Expert 

40mm 

100.0 

3 

'W 

60mm 

96.0  ! 

S3 

O 

81mm 

76.7 

.a 

u 

105mm 

100.0 

o 

X 

155mm 

100.0 

Total 

93.8 

40mm 

100.0 

*3 

60mm 

100.0 

81mm 

91.4 

tJ 

QJ 

105mm 

95.0 

> 

155mm 

97.1 

Total 

97.1 

Overall  Mean 

95.7 

2.1.5  Demographics 

Prior  to  traversing  the  test  grid,  each  participant  executed  a  basic  demographic 
questionnaire.  The  results  are  presented  as  follows: 

a.  Age.  Expert  participants  reported  ages  between  25  and  43. 

b.  Gender.  Four  of  the  five  experts  reported  being  male;  one  reported  gender  female. 

c.  Race.  Four  of  the  five  experts  reported  Caucasian;  one  reported  other. 

d.  Marital  Status.  Two  of  the  five  experts  reported  being  single;  two  reported  being 
married.  One  expert  reported  marital  status  as  divorced. 

e.  Years  of  Education.  Three  of  the  five  experts  reported  having  obtaining  a  high  school 
diploma  or  GED.  The  other  two  experts  reported  having  1  and  2  additional  years  of  education, 
respectively. 

f.  Months  of  UXO  Experience.  Expert  participants  reported  between  12  and  96  months 
of  UXO-specific  experience. 
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g.  Months  of  Schonstedt  Experience.  Expert  participants  reported  12  and  84  months  of 
Schonstedt  experience. 

h.  Months  of  EOD  Experience.  Expert  participants  reported  between  78  and  252  months 
of  EOD  experience.  The  average  amount  of  EOD  experience  was  141.6  months. 

i.  Prior  Military  Experience.  All  experts  reported  having  prior  military  experience. 

j .  Health. 

(1)  As  an  overall  indicator  of  health,  participants  were  asked  to  rate  their  current  overall 
physical  and  mental  health  status.  Choices  included  excellent,  fair,  good,  and  poor. 

(2)  Four  of  the  five  experts  reported  having  excellent  health,  whereas  one  reported  health 
status  as  good. 

k.  Use  of  Tobacco  Products.  All  expert  participants  reported  no  use  of  tobacco  products 
and  for  the  purposes  of  this  investigation  were  defined  as  nonsmokers. 

l.  Height.  Experts  reported  heights  ranging  from  64  to  72  inches. 

m.  Weight.  Experts  reported  weights  ranging  from  130  to  260  pounds. 

2.1.6  Hearing 

a.  A  certified  audiologist  performed  testing  on  all  participants.  Testing  began  with  an 
otoscopic  examination  to  detennine  the  status  of  the  pinnae  and  external  auditory  canals. 
Participants  then  completed  pure  tone  air  conduction  testing.  The  testing  took  place  in  a 
sound-treated  booth  through  the  use  of  a  clinical  diagnostic  audiometer  (Interacoustics  AC40). 
Pure  tones  at  octave  frequencies  of  250  to  8000  Hz  and  interoctaves  of  3000  and  6000  Hz  were 
presented  through  TDH-39  superaural  headphones.  Middle  ear  status  was  detennined  through 
tympanometry  using  a  Grason-Stadler  37  Auto  Tympanometer. 

b.  Overall,  hearing  tests  revealed  that,  on  average,  the  expert  participants  had  normal 
hearing  sensitivity  (defined  as  air  conduction  thresholds  of  20  dB  HL  or  better).  All  experts 
demonstrated  normal  bilateral  middle  ear  function. 

c.  One  expert  had  mild-to-moderate  high  frequency  hearing  loss  in  both  ears.  The  signals 
emitted  from  the  magnetic  locators  were  broad  in  their  frequency  spectrum,  and  therefore  should 
have  been  able  to  use  the  lower  frequency  infonnation  in  the  signal.  Indeed,  in  the  case  of  this 
expert  the  frequency  range  of  the  signal  from  500  to  3000  Hz  was  between  10  and  70  dB  above 
threshold.  Frequency  plots  are  shown  in  Figures  2.1-1  through  2.1-6. 
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Figure  2.1-1.  Plot  of  frequency  response  of  baseline  and  signals  measured  from  the  Schonstedt 
detector  along  with  hearing  thresholds  from  expert  1 . 

Note:  The  participant  has  hearing  loss  above  3000  Hz,  but  would  have  audibility  of  the  signal 
between  500  and  3000  Hz. 
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Figure  2.1-2.  Plot  of  frequency  response  of  baseline  and  signals  measured  from  the  Schonstedt 
detector  along  with  hearing  thresholds  from  expert  2. 
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Figure  2.1-3.  Plot  of  frequency  response  of  baseline  and  signals  measured  from  the  Schonstedt 
detector  along  with  hearing  thresholds  from  expert  3. 
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Figure  2-1.4.  Plot  of  frequency  response  of  baseline  and  signals  measured  from  the  Schonstedt 
detector  along  with  hearing  thresholds  from  expert  4. 
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Figure  2.1-5.  Plot  of  frequency  response  of  baseline  and  signals  measured  from  the  Schonstedt 
detector  along  with  hearing  thresholds  from  expert  5. 


Figure  2.1-6.  Average  hearing  thresholds  for  all  participants  (error  bars  indicate 

+  1  standard  deviation). 


2.1.7  Lane  Orientation  Effects 


a.  The  complete  test  grid  of  33  lanes  was  divided  into  two  groups,  corresponding  to  their 
magnetic  compass  heading. 

b.  Lanes  1  through  20  were  situated  north  to  south,  and  were  23.703  by  1.5  meters 
(35.6  m2).  There  were  28  ordnances  scattered  throughout  the  20  lanes.  The  performance  data  of 
the  expert  operators  without  instrumentation  for  lanes  1  through  20  are  presented  in  Table  2.1-3. 


TABLE  2.1-3.  SUMMARY  TABLE  OF  EXPERTS  WITHOUT  INSTRUMENTATION  FOR 

LANES  1  THROUGH  20 


Operator 

Pd 

FAR, 

1/m2 

DOP 

Lane 

Velocity, 

m/s 

Total 
Time,  s 

Experts 

E-l 

100.0% 

0.198 

0.198 

0.091 

5371 

E-2 

85.7% 

0.093 

0.170 

0.095 

5304 

E-3 

100.0% 

0.124 

0.124 

0.092 

5634 

E-4 

89.3% 

0.048 

0.117 

0.095 

5223 

E-5 

96.4% 

0.120 

0.125 

0.091 

5419 

Mean 

94.3% 

0.116 

0.147 

0.093 

5390 

Std.  Dev. 

0.0649 

0.0548 

0.0357 

0.0020 

154.9 

c.  Lanes  21  through  33  were  positioned  east  to  west.  Lanes  21  through  26  were 
17.55  meters  by  1.5  meters  (26.325  m2),  and  lanes  27  through  33  were  25.00  by  1.5  meters  (37.5 
m2).  There  were  32  ordnances  placed  throughout  the  13  lanes.  The  performance  data  without 
instrumentation  for  only  lanes  21  through  33  are  presented  in  Table  2.1-4. 


TABLE  2.1-4.  SUMMARY  TABLE  OF  ALL  OPERATORS  WITHOUT 
INSTRUMENTATION  FOR  LANES  21  THOUGH  33 


Operator 

Pd 

FAR, 

1/m2 

DOP 

Lane 

Velocity, 

m/s 

Total 
Time,  s 

Experts 

E-l 

100.0% 

0.078 

0.078 

0.101 

2867 

E-2 

96.9% 

0.036 

0.047 

0.078 

3782 

E-3 

100.0% 

0.026 

0.026 

0.111 

2645 

E-4 

96.9% 

0.010 

0.033 

0.086 

3385 

E-5 

90.6% 

0.040 

0.102 

0.103 

2780 

Mean 

96.9% 

0.038 

0.057 

0.096 

3092 

Std.  Dev. 

0.0383 

0.0255 

0.0321 

0.0133 

476.9 
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2.2  NOVICE 


2.2.1  Performance  Measurements 


a.  Pd. 

(1)  For  each  alarm  an  operator  noted  in  the  lane,  the  distance  between  each  of  the  targets 
in  the  field  and  the  alarm  was  calculated.  If  the  distance  was  less  than  1  meter,  then  a  target  was 
considered  detected  regardless  of  which  lane  the  target  was  in.  Multiple  detections  of  the  same 
target  were  ignored.  The  number  of  detected  targets  in  the  field  were  divided  by  the  total 
number  of  targets  in  the  field  (60  targets)  and  multiplied  by  100.  This  result  was  defined  as  the 
Pd  and  will  be  referred  to  as  Pd.  Pd  is  a  dimensionless  number  with  values  ranging  from  zero  to 
one. 


(2)  Novice  participants  demonstrated  Pd  rates  from  0.933  to  1.  The  average  Pd  was  0.97 
with  a  standard  deviation  of  0.0274. 

b.  FAR. 

(1)  For  each  alarm  an  operator  declared  in  a  particular  lane,  the  distance  between  each  of 
the  targets  in  the  field  and  the  alarm  was  calculated.  If  no  target  was  within  one  meter  of  the 
alarm,  then  the  alarm  was  considered  a  false  alarm.  The  total  numbers  of  false  alarms  were 
divided  by  the  area  of  the  field  (1131.5  nr).  The  result  was  defined  as  the  FAR  and  will  be 
referred  to  as  FAR.  FAR  has  a  unit  of  false  alarms  per  m  . 

(2)  Novice  participants  demonstrated  FAR  rates  from  0.017  to  0.168  false  alarms  per  m  . 
The  average  FAR  was  0.063  per  m2  with  a  standard  deviation  of  0.0605  false  alarms  per  m2. 

c.  DOP. 

(1)  An  ROC  curve  is  an  industry  standard  that  is  used  to  compare  the  performance  of 
operators  and  equipment  in  UXO  and  mine  detection.  It  consists  of  the  FAR  on  the  x-axis  versus 
the  Pd  on  the  v-axis.  Curves  nearer  the  upper  left-hand  corner  of  the  chart  are  considered  to  be 
higher  performance  from  a  detector.  Therefore,  in  order  to  compare  the  operators’  perfonnance 
versus  each  of  the  characteristics,  the  DOP  was  calculated  as  the  distance  from  the  upper 
left-hand  corner  (coordinates  0,  1)  that  an  operators’  point  (FAR,  Pd)  is  on  the  ROC  curve,  as 
shown  in  the  following  equation: 


Dist  _  DOP  =  {( l-Pd)2  +  (0-FAR y 


(2)  The  DOP  value  is  a  dimensionless  number;  however,  in  general  the  greater  the  value 
the  lower  overall  perfonnance  within  Pd  and  FAR  dimensions. 
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(3)  Novice  participants  demonstrated  DOPs  between  0.033  and  0.169.  The  average  DOP 
was  0.079  with  a  standard  deviation  of  0.0522. 

2.2.2  Time  Measurements 


a.  Lane  Velocity. 

(1)  The  time  operators  required  to  complete  each  lane  was  manually  recorded  in  an 
on-site  daily  log.  Time  delays  due  to  equipment  issues  or  data  recording  were  also  recorded  and 
then  subtracted  from  the  total  lane  time.  This  lane  length  was  then  divided  by  the  “corrected” 
lane  time.  The  result  was  defined  as  the  lane  velocity. 

(2)  Novice  participants  demonstrated  lane  velocities  between  0.050  m/s  and  0.124  m/s. 
Their  average  lane  velocity  was  0.097  m/s  with  a  standard  deviation  of  0.0284. 

b.  Total  Time. 

(1)  The  time  operators  required  to  execute  a  UXO  sweep  operation  on  all  33  lanes  of  the 
test  site.  Corrected  lane  times  were  used  for  the  following  summation: 


33 

Total  _Time  =  /  _  lanes 

i=i 


(2)  Novice  participants  demonstrated  total  times  between  8199  seconds  and 
9086  seconds.  The  average  total  times  were  8482  seconds  with  a  standard  deviation  of 
374.7  seconds. 

2.2.3  Data  Summary 


TABLE  2.2- 1 .  PERFORMANCE  OF  UN-INSTRUMENTED  NOVICE  PARTICIPANTS 


1 

Operator 

Pc 

FAR, 

1/m2 

DOP 

Lane 

Velocity, 

m/s 

Total 
Time,  s 

Novices 

N-l 

100.0% 

0.057 

0.057 

0.104 

7612 

N-2 

98.3% 

0.028 

0.033 

0.112 

6810 

N-3 

95.0% 

0.047 

0.069 

0.124 

6661 

N-4 

98.3% 

0.168 

0.169 

0.50 

17073 

N-5 

93.3% 

0.017 

0.069 

0.094 

8733 

Mean 

97.0% 

0.063 

0.079 

0.097 

9378 

Std.  Dev. 

0.0274 

0.0605 

0.0522 

0.0284 

4379.6 
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2.2.4  Ordnance.  Five  different  ordnance  types  were  used:  40mm,  60mm,  81mm,  105mm,  and 
155mm.  Each  type  was  buried  at  a  specific  depth:  6,  12,  18,  24,  and  30  inches.  The  ordnance 
was  placed  in  both  the  horizontal  and  vertical  orientations.  The  Pd  measurements  for  each 
ordnance  type,  depth,  and  orientation  are  presented  in  Table  2.2-2. 


TABLE  2.2-2.  NOVICES  WITHOUT  INSTRUMENTATION  -  Pd  BY  ORDNANCE 

ORIENTATION  AND  TYPE 


□ 

Type 

Novices 

Horizontal 

40mm 

100.0% 

60mm 

92.0% 

81mm 

83.3% 

105mm 

100.0% 

155mm 

100.0% 

Total 

94.6% 

Vertical 

40mm 

100.0% 

60mm 

97.1% 

81mm 

94.3% 

105mm 

100.0% 

155mm 

100.0% 

Total 

98.8% 

Overall  Mean 

97.0% 

2.2.5  Demographics 

Prior  to  traversing  the  test  grid,  each  participant  executed  a  basic  demographic 
questionnaire.  The  results  are  presented  as  follows: 

a.  Age.  Novice  participants  reported  ages  between  22  and  53. 

b.  Gender.  All  five  novices  reported  their  gender  as  male. 

c.  Race.  Two  of  the  five  novices  reported  their  race  as  Native  American  and  two  reported 
their  race  as  Pacific  Islander.  One  novice  reported  Caucasian. 

d.  Marital  Status.  Three  of  the  five  novices  reported  being  single;  two  reported  their 
status  as  married. 

e.  Years  of  Education.  Three  of  the  five  novices  reported  having  obtaining  a  bachelor 
degree.  The  other  two  novices  reported  having  a  high  school  education. 

f.  Months  of  UXO  Experience.  Novice  participants  reported  between  0  and  1.5  months 
of  UXO-specific  experience. 

g.  Months  of  Schonstedt  Experience.  Novice  participants  reported  between  0  and  1.5 
months  of  Schonstedt  experience. 
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h.  Months  of  EOD  Experience.  Novice  participants  reported  0  months  of  EOD 
experience. 

i.  Prior  Military  Experience.  Four  of  the  five  novices  reported  having  prior  military 
experience,  whereas  one  novice  reported  no  military  experience. 

j .  Health. 

(1)  As  an  overall  indicator  of  health,  participants  were  asked  to  rate  their  current  overall 
physical  and  mental  health  status.  Choices  included  excellent,  fair,  good,  and  poor. 

(2)  All  five  novices  reported  having  good  health. 

k.  Use  of  Tobacco  Products.  Four  of  the  five  novice  participants  reported  no  use  of 
tobacco  products  and  for  the  purposes  of  this  investigation  were  defined  as  nonsmokers.  One 
novice  reported  the  use  of  tobacco  products  (cigarettes)  at  a  rate  of  1  pack  per  day. 

l.  Height.  Novices  reported  heights  ranging  from  67  to  73  inches. 

m.  Weight.  Novices  reported  weights  ranging  from  160  to  230  pounds. 

2.2.6  Hearing 

a.  A  certified  audiologist  performed  testing  on  all  participants.  Testing  began  with  an 
otoscopic  examination  to  detennine  the  status  of  the  pinnae  and  external  auditory  canals. 
Participants  then  completed  pure  tone  air  conduction  testing.  The  testing  took  place  in  a 
sound-treated  booth  through  the  use  of  a  clinical  diagnostic  audiometer  (Interacoustics  AC40). 
Pure  tones  at  octave  frequencies  of  250  to  8000  Hz  and  interoctaves  of  3000  and  6000  Hz  were 
presented  through  TDH-39  superaural  headphones.  Middle  ear  status  was  determined  through 
tympanometry  using  a  Grason-Stadler  37  Auto  Tympanometer. 

b.  Overall,  hearing  tests  revealed  that,  on  average,  the  novice  participants  had  normal 
hearing  sensitivity  (defined  as  air  conduction  thresholds  of  20  dB  HL  or  better).  All  novices 
demonstrated  normal  bilateral  middle  ear  function. 

c.  Two  additional  participants,  novices  2  and  4,  had  moderate  hearing  loss  in  the  high 
frequencies,  but  only  in  one  ear.  In  novice  2,  the  hearing  loss  was  in  the  left  ear;  for  novice  4  the 
loss  was  in  the  right  ear.  Frequency  plots  are  shown  in  Figures  2.1-1  through  2.1-6. 
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Figure  2.2-1.  Plot  of  frequency  response  of  baseline  and  signals  measured  from  the  Schonstedt 
detector  along  with  hearing  thresholds  from  novice  1 . 
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Figure  2.2-2.  Plot  of  frequency  response  of  baseline  and  signals  measured  from  the  Schonstedt 
detector  along  with  hearing  thresholds  from  novice  2. 

Note:  The  participant  has  a  hearing  loss  in  the  high  frequencies  in  the  left  ear,  as  shown  by  the  x 
symbols. 
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Figure  2.2-3.  Plot  of  frequency  response  of  baseline  and  signals  measured  from  the  Schonstedt 
detector  along  with  hearing  thresholds  from  novice  3. 
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Figure  2.2-4.  Plot  of  frequency  response  of  baseline  and  signals  measured  from  the  Schonstedt 
detector  along  with  hearing  thresholds  from  novice  4. 

Note:  The  participant  has  a  high  frequency  hearing  loss  in  the  right  ear,  as  shown  by  the  open 
circles. 
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Figure  2.2-5.  Plot  of  frequency  response  of  baseline  and  signals  measured  from  the  Schonstedt 
detector  along  with  hearing  thresholds  from  novice  5. 


2.2.7  Lane  Orientation  Effects 


a.  The  complete  test  grid  of  33  lanes  was  divided  into  two  magnetic  compass  headings 
sections. 

b.  Lanes  1  through  20  was  situated  north  to  south,  and  was  23.703  by  1.5  meters 
(35.6  m  ).  There  were  28  ordnances  scattered  throughout  the  20  lanes.  The  performance  data  of 
the  novice  operators  without  instrumentation  for  lanes  1  through  20  are  presented  in  Table  2.2-3. 


TABLE  2.2-3.  SUMMARY  TABLE  OF  ALL  OPERATORS  WITHOUT 
INSTRUMENTATION  FOR  LANES  1  THROUGH  20 


1 

Operator 

Pd 

FAR, 

1/m2 

DOP 

Lane 

Velocity, 

m/s 

Total 
Time,  s 

Novices 

N-l 

100.0% 

0.083 

0.083 

0.101 

4916 

N-2 

100.0% 

0.042 

0.042 

0.109 

4400 

N-3 

96.4% 

0.055 

0.065 

0.141 

3685 

N-4 

100.0% 

0.167 

0.167 

0.060 

8498 

N-5 

92.9% 

0.023 

0.075 

0.112 

4475 

Mean 

97.9% 

0.074 

0.087 

0.105 

5195 

Std.  Dev. 

0.0319 

0.0566 

0.0477 

0.0292 

1898.6 
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c.  Lanes  21  through  33  were  positioned  east  to  west.  Lanes  21  through  26  were  17.55  by 
1.5  meters  (26.325  m2),  and  lanes  27  through  33  were  25.00  by  1.5  meters  (37.5  nr).  There  were 
32  ordnances  placed  throughout  the  13  lanes.  The  performance  data  without  instrumentation  for 
only  lanes  21  through  33  are  presented  in  Table  2.2-4. 


TABLE  2.2-4.  SUMMARY  TABLE  OF  ALL  OPERATORS  WITHOUT 
INSTRUMENTATION  FOR  LANES  21  THROUGH  33 


1 

Pd 

FAR, 

1/m2 

DOP 

Lane 

Velocity, 

m/s 

Total 
Time,  s 

Novices 

N-l 

100.0% 

0.012 

0.018 

2696 

N-2 

96.9% 

0.005 

0.117 

2410 

N-3 

93.8% 

0.033 

0.098 

2976 

N-4 

96.9% 

0.169 

0.172 

0.035 

8575 

N-5 

93.8% 

0.007 

0.063 

0.068 

4258 

Mean 

96.3% 

0.045 

0.070 

0.085 

4183 

Std.  Dev. 

0.0261 

0.0701 

0.617 

0.0338 

2554.7 
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2.3  DISCUSSION  OF  UN-INSTRUMENTED  DATA 


2.3.1  Performance  Analysis 

a.  A  summary  of  the  performance  data  for  each  expert  and  novice  without  instrumentation 
is  presented  in  Table  2.3-1.  The  Pd  versus  FAR  of  experts  and  novices  is  shown  in  Figure  2.3-1. 


TABLE  2.3- 1 .  SUMMARY  OF  PERFORMANCE  DATA  OF  EXPERTS  AND  NOVICES 

WITHOUT  INSTRUMENTATION 


Operator 

Probability 
of  Detection 
(Pd) 

False 

Alarm 

Rate 

(1/mA2) 

DOP 

Lane 

Velocity 

(m/s) 

Total 
Time  (s) 

Experts 

E-1 

100.0% 

0.154 

0.154 

0.095 

8238 

E-2 

91.7% 

0.072 

0.110 

0.088 

9086 

E-3 

100.0% 

0.087 

0.087 

0.100 

8279 

E-4 

93.3% 

0.034 

0.075 

0.092 

8608 

E-5 

93.3% 

0.090 

0.112 

0.096 

8199 

Mean 

95.7% 

0.087 

0.108 

0.094 

8482 

Std.  Dev. 

0.0401 

0.0435 

0.0302 

0.0044 

374.7 

Novices 

N-1 

100.0% 

0.057 

0.057 

0.104 

7612 

N-2 

98.3% 

0.028 

0.033 

0.112 

6810 

N-3 

95.0% 

0.047 

0.069 

0.124 

6661 

N-4 

98.3% 

0.168 

0.169 

0.050 

17073 

N-5 

93.3% 

0.017 

0.069 

0.094 

8733 

Mean 

97.0% 

0.063 

0.079 

0.097 

9378 

Std.  Dev. 

0.0274 

0.0605 

0.0522 

0.0284 

4379.6 

Figure  2.3-1 .  Pd  versus  FAR  of  experts  and  novices  without  instrumentation. 
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b.  Using  the  chi-square  distribution  at  the  0.05  significance  level,  Pd  between  experts  and 
novices,  without  instrumentation,  was  not  found  to  be  significantly  different.  Using  the 
Mann- Whitney  test  at  the  0.05  significance  level,  no  significant  differences  were  found  between 
the  number  of  false  alarms,  DOP,  and  time  between  the  novices  and  experts. 

c.  The  comparison  of  average  lane  velocity  with  the  three  performance  measurements  (Pd, 
FAR,  and  DOP)  is  shown  in  Figures  2.3-2  through  2.3-4. 

d.  The  comparison  of  total  time  with  the  three  performance  measurements  is  shown  in 
Figures  2.3-5  through  2.3-7. 
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Figure  2.3-2.  Experts  and  novices  -  Pd  versus  average  lane  velocity. 
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Figure  2.3-3.  Experts  and  novices  -  FAR  versus  average  lane  velocity. 
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Figure  2.3-4.  Experts  and  novices  -  DOP  versus  average  lane  velocity. 
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Figure  2.3-5.  Experts  and  novices  -  Pd  versus  total  time. 
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Figure  2.3-6.  Experts  and  novices  -  FAR  versus  total  time. 
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Figure  2.3-7.  Experts  and  novices  -  DOP  versus  total  time. 


2.3.2  Ordnance 


a.  The  average  Pd  for  each  ordnance  type/depth  and  orientation  for  all  the  operators 
without  instrumentation  is  presented  in  Table  2.3-2.  All  operators  achieved  100  percent 
detection  rates  on  the  40mm,  both  horizontally  and  vertically  oriented.  Operators  had  the  lowest 
detection  rates  on  the  81mm,  both  horizontally  and  vertically  oriented.  Overall,  vertically 
oriented  ordnance  had  higher  Pd  than  horizontally  oriented  ordnance.  Ordnance  type  and  depths 
were  codependent  (table  2.3-2);  that  is,  since  each  ordnance  type  was  buried  at  a  certain  depth, 
no  infonnation  can  be  deduced  about  ordnance  type  or  depth  separately. 
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TABLE  2.3-2.  AVERAGE  Pd  OF  ALL  OPERATORS  WITHOUT  INSTRUMENTATION  BY 
ORDNANCE  TYPE/DEPTH  AND  ORIENTATION 


Type 

Depth,  in 

Horizontal 

Vertical 

Total 

40mm 

6 

100.0% 

100.0% 

100.0% 

60mm 

12 

94.0% 

98.6% 

96.7% 

81mm 

18 

80.0% 

92.9% 

88.5% 

105mm 

24 

100.0% 

97.5% 

98.5% 

155mm 

30 

100.0% 

98.6% 

99.2% 

Total 

94.2% 

97.9% 

96.3% 

b.  The  Pd  by  ordnance  orientation  and  type  for  both  novices  and  experts  is  presented  in 
Table  2.3-3.  Novices  had  100  percent  Pd  both  horizontally  and  vertically  with  three  different 
types  of  ordnance:  40mm,  105mm,  and  155mm.  They  also  performed  equal  to  or  better  than  the 
experts  in  every  category  with  the  exception  of  two,  the  60mm  horizontally  and  vertically, 
overall,  the  novices  averaged  1 . 1  percent  higher  with  regard  to  Pd  than  the  experts. 


TABLE  2.3-3.  EXPERTS  VERSUS  NOVICES  WITHOUT  INSTRUMENTATION  -  Pd  BY 

ORDNANCE  ORIENTATION  AND  TYPE 


n 

Type 

Novices 

Experts 

Total 

Pd  Differences 

Horizontal 

40mm 

100.0% 

100.0% 

100.0% 

0.0%  | 

60mm 

92.0% 

96.0% 

94.0% 

-4.0% 

81mm 

83.3% 

76.7% 

80.0% 

6.7% 

105mm 

100.0% 

100.0% 

100.0% 

0.0% 

155mm 

100.0% 

100.0% 

100.0% 

0.0% 

Total 

94.6% 

93.8% 

94.2% 

Vertical 

40mm 

100.0% 

100.0% 

100.0% 

0.0% 

60mm 

97.1% 

100.0% 

98.6% 

-2.9% 

81mm 

94.3% 

91.4% 

92.9% 

2.9% 

105mm 

100.0% 

95.0% 

97.5% 

5.0% 

155mm 

100.0% 

97.1% 

98.6% 

2.9% 

Total 

98.8% 

97.1% 

97.9% 

Overall  Total  97.0%  95.7%  96.3% 

2.3.3  Audiological  Analysis.  For  all  participants,  hearing  status  was  not  expected  to  have  a 
significant  impact  on  performance  in  the  study  for  three  reasons.  First,  the  signals  from  the 
magnetic  locators  presented  to  the  listeners  were  provided  through  headphones  to  both  ears. 
Therefore,  hearing  loss  in  only  one  ear  should  not  impact  overall  performance,  as  the  better  ear 
would  be  able  to  compensate  for  the  loss  in  the  poorer  ear.  Second,  the  signals  provided  to  the 
listeners  were  of  a  level  high  enough  to  be  above  their  hearing  thresholds,  as  shown  in  the 
graphs.  Third,  the  signals  emitted  from  the  magnetic  locators  were  broad  in  their  frequency 
spectrum,  so  that  an  individual  with  a  bilateral  high  frequency  hearing  loss  should  have  been  able 
to  use  the  lower  frequency. 
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2.3.4  Lane  Orientation  Analysis.  A  statistical  analysis  was  done  to  compare  perfonnance  data 
from  lanes  1  through  20  (north-south  lanes)  with  the  data  from  lanes  21  through  33  (east- west 
lanes).  Pd  was  tested  using  the  chi-square  test  at  the  0.05  significance  level,  while  the  number  of 
false  alanns,  distance  measurement,  and  total  time  were  tested  using  the  Mann- Whitney  test  at 
the  0.05  significance  level.  The  perfonnance  measurements  that  were  significantly  different 
between  experts  and  novices  are  presented  in  Table  2.3-4.  Without  instrumentation  for  all  lanes, 
the  novices  and  experts  performed  similarly.  The  significant  difference  between  lanes  1  through 
20  data  and  lanes  21  through  33  data  (total  time  was  not  addressed  in  these  comparisons  because 
the  area  covered  was  not  the  same)  are  presented  in  Table  2.3-5.  For  experts  and  novices  both 
with  and  without  instrumentation,  significantly  more  false  alarms  were  found  for  lanes  1  through 
20  than  lanes  21  through  33.  Common  false  alarms  (CFA)  are  false  alarms  in  which  four  or 
more  operators  detected  a  FAR  within  a  0.25-meter  radius  circle.  For  the  operators  without 
instrumentation  (78  CFA),  86  percent  occurred  in  lanes  1  through  20. 


TABLE  2.3-4.  SUMMARY  OF  SIGNIFICANCE  TESTING  OF  EXPERTS  VERSUS 

NOVICES 


Pda 

#FAb 

DOPb 

with 

Inst 

Lanes  1-33 

SIG 

— 

SIG 

Lanes  1-20 

SIG 

SIG 

SIG 

Lanes  21-33 

SIG 

— 

SIG 

without 

Inst 

Lanes  1-33 

— 

— 

— 

Lanes  1-20 

— 

— 

SIG 

Lanes  21-33 

-- 

- 

- 

aChi-square  distribution,  one-sided  test,  at  0.05  significance  level. 
bMann- Whitney  test,  one-sided  test,  at  0.05  significance  level. 


TABLE  2.3-5.  SUMMARY  OF  SIGNIFICANCE  TESTING  OF  DATA  FROM  LANES  1 
THROUGH  20  VERSUS  LANES  21  THROUGH  33 


Pda 

#FAb 

DOPb 

with 

Experts 

— 

SIG 

— 

Inst 

Novices 

— 

SIG 

— 

without 

Experts 

— 

SIG 

SIG 

Inst 

Novices 

- 

SIG 

- 

aChi-square  distribution,  one-sided  test,  at  0.05  significance  level. 
bMann- Whitney  test,  one-sided  test,  at  0.05  significance  level. 


2.3.5  Stress  Analysis.  See  appendix  M. 
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2.3.6  Observations 


a.  Overall,  the  novices  had  higher  Pd  and  lower  FAR  within  the  un-instrumented 
subgroup.  Consequently,  it  was  not  unexpected  that  DOPs  for  the  novices  were  also  lower,  on 
average,  than  those  for  the  experts.  It  is  also  interesting  to  observe  that  despite  the  overall 
performance  averages,  two  of  the  experts  had  a  Pd  of  one,  versus  only  one  novice.  The  Pd  versus 
FAR  figure  shows  that  for  the  most  part,  novices  tended  to  be  closer  to  the  upper  left-hand  corner 
and  experts  were  grouped  more  to  the  right.  This  matches  the  results  from  the  DOP  calculations 
shown  in  Table  2.3-1. 

b.  Two  trends  can  be  seen  from  the  expert  versus  novice  comparison  plots 
(fig.  2.3-2  through  2.3-7):  the  quicker  the  novices  traversed  the  lanes,  the  fewer  false  alanns 
they  indicated,  and  they  also  had  a  better  overall  perfonnance  (low  DOP).  Also,  the  experts  had 
little  variation  among  themselves  for  lane  velocity  (standard  deviation,  0.0044).  The  novices,  on 
the  other  hand,  had  high  variation  for  lane  velocity  (standard  deviation,  0.0284). 

c.  When  reviewing  perfonnance  characteristics,  some  observations  were  clear  with  regard 
to  time  domain  data  in  Figures  2.3-5  to  2.3-7.  Consistent  with  average  lane  velocity,  the  experts 
had  less  variation  among  themselves  then  the  novices  for  total  time.  Also,  the  more  time  the 
novices  took,  the  more  false  alanns  they  indicated. 

d.  Overall,  vertically  oriented  ordnance  had  higher  Pd  than  the  horizontally  oriented 
ordnance.  Both  novices  and  experts  had  difficulty  in  locating  the  61mm  and  81mm  mortar 
targets.  This  may  have  been  because  of  the  distinctive  profile  of  the  latter  targets  compared  with 
the  40mm,  105mm,  and  155mm  targets,  which  share  a  simple  projectile  geometry.  It  was 
unexpected  that  both  experts  and  novices  located  (Pd  rates  of  1)  the  40mm  projectile  despite  the 
predominantly  aluminum  alloy  composition,  albeit  this  target  was  positioned  at  a  mere  6-inch 
burial  depth. 

e.  Experts  showed  an  overall  lower  average  total  time  of  2.4  hours,  whereas  the  novices 
reported  an  average  total  time  of  2.6  hours.  Novice  4  is  an  outlier  in  this  data  set  and  accounts 
for  the  difference  in  average  total  time.  It  is  assumed  that  if  novice  4  performed  in  a  similar 
manner  to  his  novice  counterparts,  this  average  would  be  lower.  Novice  4’s  time  perfonnance 
also  accounts  for  the  considerably  large  standard  deviation  within  the  novice  data  set  of 
1.2  hours. 

f.  The  demographics  of  experts  and  novices  seemed  to  provide  a  wide  range  of 
individuals.  Ages,  races,  and  marital  status  seemed  to  vary  with  no  conclusive  pattern.  The 
majority  of  the  participants  (9  of  10)  were  male,  with  only  one  female  in  the  expert  group. 
Education  levels  seemed  to  be  higher  in  the  novice  group,  with  two  participants  having  obtained 
a  bachelor  degree.  The  experts  all  held  high  school  diplomas,  with  1  or  2  years  of  additional 
study.  It  is  possible  that  expert  participants  interpreted  their  military  training  as  education,  and 
this  may  account  for  the  differences.  All  experts,  and  4  out  of  5  novices,  reported  prior  military 
experience.  Of  that  military  experience,  the  experts  had  an  average  of  1 1.8  years  of  military,  4.5 
years  of  UXO,  and  2.3  years  of  Schonstedt  specific  experience.  One  novice  reported  1.5  months 
of  UXO  and  Schonstedt  experience;  however,  this  was  assumed  to  be  a  subjective  answer  as  he 
had  graduated  from  the  UXO  program  1.5  months  before  participation  in  this  study.  In  addition, 
this  novice  did  not  work  on  a  UXO  site  between  graduation  from  the  UXO  training  program  and 
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participation  in  this  study.  It  was  determined  that  these  data  would  be  valid  for  inclusion  for 
analysis. 


g.  The  heaviest  expert  was  consistent  with  the  individual  greatest  in  height,  and  the 
lightest  expert  was  the  shortest.  No  physical  outliers  were  observed  in  either  set. 

h.  It  is  interesting  that  most  experts  (four  of  five)  reported  having  excellent  health, 
whereas  all  novices  reported  having  good  health.  All  experts  were  nonsmokers,  and  only  one 
novice  reported  using  tobacco  products. 
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SECTION  3.  INSTRUMENTED  DATA 


3.1  EXPERT 

3.1.1  Performance  Measurements 


a.  Pd. 

(1)  For  each  alarm  an  operator  noted  in  the  lane,  the  distance  between  each  of  the  targets 
in  the  field  and  the  alarm  was  calculated.  If  the  distance  was  less  than  1  meter,  then  a  target  was 
considered  detected  regardless  of  which  lane  the  target  was  in.  Multiple  detections  of  the  same 
target  were  ignored.  The  number  of  detected  targets  in  the  field  were  divided  by  the  total 
number  of  targets  in  the  field  (60  targets)  and  multiplied  by  100.  This  result  was  defined  as  the 
Pd  and  will  be  referred  to  as  Pd.  Pd  is  a  dimensionless  number  with  values  ranging  from  zero  to 
one. 


(2)  Expert  participants  demonstrated  Pd  rates  from  0.617  to  1.  Their  average  Pd  was 
0.877  with  a  standard  deviation  of  0.0159. 

b.  FAR. 

(1)  For  each  alarm  an  operator  declared  in  a  particular  lane,  the  distance  between  each  of 
the  targets  in  the  field  and  the  alarm  was  calculated.  If  no  target  was  within  1  meter  of  the  alarm, 
then  the  alarm  was  considered  a  false  alarm.  The  total  numbers  of  false  alarms  were  divided  by 
the  area  of  the  field  (1 131.5  nr).  The  result  was  defined  as  the  FAR  and  will  be  referred  to  as 
FAR.  FAR  has  a  unit  of  false  alarms  per  m  . 

(2)  Expert  participants  demonstrated  FAR  rates  from  0.012  to  0.96  false  alanns  per  m  . 
The  average  FAR  was  0.061  per  m2  with  a  standard  deviation  of  0.0333  false  alarms  per  m2. 

c.  DOP. 

(1)  An  ROC  curve  is  an  industry  standard  that  is  used  to  compare  the  performance  of 
operators  and  equipment  in  UXO  and  mine  detection.  It  consists  of  the  FAR  on  the  x-axis  versus 
the  Pd  on  the  y-axis.  Curves  nearer  the  upper  left-hand  corner  of  the  chart  are  considered  to  be 
higher  performance  from  a  detector.  Therefore,  in  order  to  compare  the  operators’  perfonnance 
versus  each  of  the  characteristics,  the  DOP  was  calculated  as  the  distance  from  the  upper 
left-hand  corner  (coordinates  0,  1)  that  an  operators’  point  (FAR,  Pd)  is  on  the  ROC  curve,  as 
shown  in  the  following  equation: 


Dist  _  DOP  =  j(l-Pd)2  +  (0-FAR)2 
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(2)  The  DOP  value  is  a  dimensionless  number;  however,  in  general  the  greater  the  value 
the  lower  overall  performance  within  Pd  and  FAR  dimensions.  Expert  participants  demonstrated 
DOP’s  between  0.087  and  0.384.  Their  average  DOP  was  0.161  with  a  standard  deviation  of 
0.1325. 

3.1.2  Time  Measurements 


a.  Lane  velocity. 

(1)  The  time  operators  required  to  complete  each  lane  was  manually  recorded  in  an 
on-site  daily  log.  Time  delays  due  to  equipment  issues  or  data  recording  were  also  recorded  and 
then  subtracted  from  the  total  lane  time.  This  lane  length  was  then  divided  by  the  “corrected” 
lane  time.  The  result  was  defined  as  the  lane  velocity. 

(2)  Expert  participants  demonstrated  lane  velocities  between  0.069  m/s  and  0.105  m/s. 
Their  average  lane  velocity  was  0.082  m/s  with  a  standard  deviation  of  0.0147. 

b.  Total  time. 

(1)  The  time  operators  required  to  execute  a  UXO  sweep  operation  on  all  33  lanes  of  the 
test  site.  Corrected  lane  times  were  used  for  the  following  summation: 


33 


Total  _  Time  =  J'j  _  lanes 


i= 1 


(2)  Expert  participants  demonstrated  total  times  between  7574  seconds  and 
1 1,626  seconds.  Their  average  total  times  were  9953  seconds  with  a  standard  deviation  of 
1565.4  seconds. 


3.1.3  Data  Summary 


TABLE  3.1-1.  EXPERT  PERFORMANCE  DATA  (INSTRUMENTED) 


1 

Operator 

Pd 

FAR, 

1/m2 

DOP 

Lane 

Velocity, 

m/s 

Total 

Times 

Experts 

E-l 

98.3% 

0.048 

0.051 

0.089 

9304 

E-2 

83.3% 

0.061 

0.177 

0.075 

10602 

E-3 

100.0% 

0.087 

0.087 

0.074 

10661 

E-4 

61.7% 

0.012 

0.384 

0.105 

7574 

E-5 

95.0% 

0.096 

0.109 

0.069 

11626 

Mean 

87.7% 

0.061 

0.161 

0.082 

9953 

Std.  Dev. 

0.1593 

0.0333 

0.1325 

0.0147 

1565.4 
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3.1.4  Ordnance.  Five  different  ordnance  types  were  used:  40mm,  60mm,  81mm,  105mm,  and 
155mm.  Each  type  was  buried  at  a  specific  depth:  6,  12,  18,  24,  and  30  inches.  The  ordnance 
was  placed  in  both  the  horizontal  and  vertical  orientations.  The  Pd  measurements  for  each 
ordnance  type,  depth,  and  orientation  are  presented  in  Table  3.1-2. 


TABLE  3.1-2.  Pd  B Y  ORDNANCE  ORIENTATION  AND  TYPE 


Type 

Experts 

40mm 

100.0% 

TZ 

'W 

S3 

60mm 

80.0% 

O 

#N 

81mm 

70.0% 

© 

sc 

105mm 

80.0% 

155mm 

100.0% 

Total 

85.4% 

40mm 

100.0% 

60mm 

94.3% 

C3 

81mm 

71.4% 

u 

© 

> 

105mm 

100.0% 

155mm 

82.9% 

Total 

89.4% 

3.1.5  Demographics 

a.  Prior  to  traversing  the  test  grid,  each  participant  executed  a  basic  demographic 
questionnaire. 

b.  The  group  of  expert  participants,  when  comparing  un-instrumented  with  instrumented 
samples,  was  identical  and  contained  the  same  personnel.  Demographics  of  expert  participants 
can  be  found  in  section  2.2.5. 

3.1.6  Hearing.  The  group  of  expert  participants,  when  comparing  un-instrumented  with 
instrumented  samples,  was  identical  and  contained  the  same  personnel.  Hearing  assessment 
results  of  expert  participants  can  be  found  in  section  2.2.4. 

3.1.7  Lane  Orientation  Effects 


a.  The  complete  test  grid  of  33  lanes  was  divided  into  two  groups,  corresponding  to  their 
magnetic  compass  heading. 

b.  Lanes  1  through  20  was  situated  north  to  south,  and  was  23.703  by  1.5  meters 
(35.6  m2).  There  were  28  ordnances  scattered  throughout  the  20  lanes.  The  performance  data  of 
the  expert  operators  with  instrumentation  of  lanes  1  through  20  are  presented  in  Table  3.1-3. 
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TABLE  3.1-3.  SUMMARY  TABLE  OF  ALL  OPERATORS  WITH  INSTRUMENTATION 

FOR  LANES  1  THROUGH  20 


Operator 

Pd 

FAR, 

1/m2 

DOP 

Lane 

Velocity, 

m/s 

Total 
Time,  s 

Experts 

E-l 

100.0% 

0.052 

0.052 

0.101 

4904 

E-2 

82.1% 

0.082 

0.196 

0.068 

7259 

E-3 

100.0% 

0.107 

0.107 

0.070 

7030 

E-4 

60.7% 

0.020 

0.393 

0.094 

5247 

E-5 

92.9% 

0.136 

0.154 

0.062 

7907 

Mean 

87.1% 

0.079 

0.181 

0.079 

6469 

Std.  Dev. 

0.1648 

0.0456 

0.1306 

0.0172 

1318.1 

c.  Lanes  21  through  33  was  positioned  east  to  west.  Lanes  21  through  26  were  17.55  by 
1.5  meters  (26.325  m2),  and  lanes  27  through  33  were  25.00  by  1.5  meters  (37.5  m2).  There  were 
32  ordnances  placed  throughout  these  thirteen  lanes.  The  performance  data  without 
instrumentation  for  only  lanes  21  through  33  are  presented  in  Table  3.1-4. 


TABLE  3.1-4.  SUMMARY  TABLE  OF  ALL  OPERATORS  WITH  INSTRUMENTATION 

FOR  LANES  21  THOUGH  33 


1 

Operator 

Pd 

FAR, 

1/m2 

DOP 

Lane 

Velocity, 

m/s 

Total 
Time,  s 

Experts 

E-l 

96.9% 

0.040 

0.051 

0.070 

4400 

E-2 

84.4% 

0.026 

0.158 

0.085 

3343 

E-3 

100.0% 

0.052 

0.052 

0.081 

3631 

E-4 

62.5% 

0.000 

0.375 

0.121 

2327 

E-5 

96.9% 

0.029 

0.042 

0.079 

3719 

Mean 

88.1% 

0.029 

0.136 

0.087 

3484 

Std.  Dev. 

0.1553 

0.0195 

0.1420 

0.0198 

754.1 

3.1.8  Dynamic  Measurements 

a.  Percentage  of  lane  area  covered. 

(1)  Using  the  TMS  data,  the  lateral  distance  between  the  detector  head  and  each  point  on 
a  0.25  meter  square  grid  within  the  lane  was  calculated  for  each  recorded  coordinate  of  the 
detector  head.  The  number  of  points  on  the  grid  of  which  the  detector  head  came  within 
0.25  meters  at  some  point  during  the  run  was  divided  by  the  total  number  of  points  on  the  grid 
and  multiplied  by  100.  The  result  was  defined  as  the  percent  of  lane  area  covered. 

(2)  Experts  were  observed  to  have  between  92.82  and  98.36  percent  lane  coverage  rates. 
The  average  and  standard  deviation  was  96.50  percent  and  2.1  percent  coverage  rates, 
respectively. 
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b.  Detector  head  height. 


(1)  Using  the  TMS  data,  the  two  sensors’  positions  were  used  to  calculate  a  vector  to 
determine  the  position  of  the  detector  head.  The  ground  altitude  at  the  nearest  surveyed  point 
was  then  subtracted  from  the  altitude  of  the  calculated  detector  head  position.  The  result  was 
defined  as  the  detector  head  height.  The  data  presented  are  considered  to  be  an  average,  as  they 
were  compared  with  the  time-stamped  lane  data. 

(2)  Experts  were  observed  with  average  detector  head  heights  between  7.09  and  9.79 
inches.  The  average  and  standard  deviation  was  8.48  and  1.31 1  inches,  respectively. 

c.  Detector  head  velocity. 

(1)  Using  the  TMS  data,  the  incremental  distance  traveled  by  the  detector  head  was 
calculated  by  taking  the  calculated  detector  head  position  at  each  instance  and  subtracting  the 
calculated  detector  head  position  at  the  previous  instance.  The  incremental  distance  traveled  was 
then  divided  by  the  time  lapse  (normally  0.1  s).  The  result  was  defined  as  the  detector  head 
velocity. 

(2)  Experts  were  observed  with  detector  head  velocities  between  1.38  and  2.62  m/s.  The 
average  and  standard  deviation  were  1.84  and  0.463  m/s,  respectively. 


TABLE  3.1-5.  SUMMARY  OF  EXPERT  ’  S  DYNAMIC  DATA 


Experts 

Pd 

FAR, 

1/m2 

DOP 

Distance 

Lane 

Velocity, 

m/s 

%  Lane 
Area 
Covered 

Detector 

Head 

Height, 

in. 

Detector 

Head 

Velocity, 

m/s 

E-l 

98.33% 

0.0477 

0.0506 

0.09 

97.13% 

7.09 

1.38 

E-2 

83.33% 

0.0610 

0.1775 

0.07 

92.82% 

8.88 

1.74 

E-3 

100.00% 

0.0875 

0.0875 

0.07 

97.26% 

7.10 

1.73 

E-4 

61.67% 

0.0124 

0.3835 

0.10 

98.36% 

9.79 

2.62 

E-5 

95.00% 

0.0963 

0.1085 

0.06 

96.95% 

9.56 

1.72 

Mean 

87.67% 

0.0610 

0.1615 

0.08 

96.50% 

8.48 

1.84 

St.  Dev. 

0.159 

0.034 

0.132 

0.016 

0.021 

1.311 

0.463 
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3.2  NOVICE 


3.2.1  Performance  Measurements 


a.  Pd. 

(1)  For  each  alarm  an  operator  noted  in  the  lane,  the  distance  between  each  of  the  targets 
in  the  field  and  the  alarm  was  calculated.  If  the  distance  was  less  than  1  meter,  then  a  target  was 
considered  detected  regardless  of  which  lane  the  target  was  in.  Multiple  detections  of  the  same 
target  were  ignored.  The  number  of  detected  targets  in  the  field  were  divided  by  the  total 
number  of  targets  in  the  field  (60  targets)  and  multiplied  by  100.  This  result  was  defined  as  the 
Pd  and  will  be  referred  to  as  Pd.  Pd  is  a  dimensionless  number  with  values  ranging  from  zero  to 
one. 


(2)  Novice  participants  demonstrated  Pd  rates  from  0.950  to  1.  Their  average  Pd  was 
0.983  with  a  standard  deviation  of  0.020. 

b.  FAR. 

(1)  For  each  alarm  an  operator  declared  in  a  particular  lane,  the  distance  between  each  of 
the  targets  in  the  field  and  the  alarm  was  calculated.  If  no  target  was  within  1  meter  of  the  alarm, 
then  the  alarm  was  considered  a  false  alarm.  The  total  numbers  of  false  alarms  were  divided  by 
the  area  of  the  field  (1 131.5  nr).  The  result  was  defined  as  the  FAR  and  will  be  referred  to  as 
FAR.  FAR  has  a  unit  of  false  alarms  per  m  . 

(2)  Novice  participants  demonstrated  FAR  rates  from  0.013  to  0.42  false  alarms  per  m  . 
The  average  FAR  was  0.021  per  m  with  a  standard  deviation  of  0.014  false  alanns  per  m  . 

c.  DOP. 

(1)  An  ROC  curve  is  an  industry  standard  that  is  used  to  compare  the  performance  of 
operators  and  equipment  in  UXO  and  mine  detection.  It  consists  of  the  FAR  on  the  x-axis  versus 
the  Pd  on  the  v-axis.  Curves  nearer  the  upper  left-hand  corner  of  the  chart  are  considered  to  be 
higher  performance  from  a  detector.  Therefore,  in  order  to  compare  the  operators’  perfonnance 
versus  each  of  the  characteristics,  the  DOP  was  calculated  as  the  distance  from  the  upper 
left-hand  corner  (coordinates  0,  1)  that  an  operators’  point  (FAR,  Pd)  is  on  the  ROC  curve,  as 
shown  in  the  following  equation: 


Dist  _  DOP  =  l-Pd)2  +  (0-FAR y 
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(2)  The  DOP  value  is  a  dimensionless  number;  however,  in  general  the  greater  the  value 
the  lower  overall  perfonnance  within  Pd  and  FAR  dimensions.  Novice  participants  demonstrated 
DOP’s  between  0.015  and  0.052.  Their  average  DOP  was  0.032  with  a  standard  deviation  of 
0.0155. 

3.2.2  Time  Measurements 


a.  Lane  Velocity. 

(1)  The  time  operators  required  to  complete  each  lane  was  manually  recorded  in  an 
on-site  daily  log.  Time  delays  due  to  equipment  issues  or  data  recording  were  also  recorded  and 
then  subtracted  from  the  total  lane  time.  This  lane  length  was  then  divided  by  the  “corrected” 
lane  time.  The  result  was  defined  as  the  lane  velocity. 

(2)  Novice  participants  demonstrated  lane  velocities  between  0.035  m/s  and  0.11  m/s. 
Their  average  lane  velocity  was  0.087  m/s  with  a  standard  deviation  of  0.0302. 

b.  Total  time. 

(1)  The  time  operators  required  to  execute  a  UXO  sweep  operation  on  all  33  lanes  of  the 
test  site.  Corrected  lane  times  were  used  for  the  following  summation: 
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Total  _  Time  =  J'j  _  lanes 


i= 1 


(2)  Novice  participants  demonstrated  total  times  between  7074  seconds  and 
23,347  seconds.  Their  average  total  times  were  1 1,186  seconds  with  a  standard  deviation  of 

6851.3  seconds. 

3.2.3  Data  Summary 


TABLE  3.2-1.  SUMMARY  OF  PERFORMANCE  DATA  OF  NOVICES  WITH 

INSTRUMENTATION 


1 

Operator 

Pd 

FAR, 

1/m2 

DOP 

Lane 

Velocity, 

m/s 

Total 
Time,  s 

Novices 

N-l 

98.3% 

0.020 

0.026 

0.088 

8848 

N-2 

98.3% 

0.013 

0.021 

0.110 

7074 

N-3 

95.0% 

0.016 

0.052 

0.097 

9097 

N-4 

100.0% 

0.042 

0.042 

0.035 

23347 

N-5 

100.0% 

0.015 

0.015 

0.106 

7562 

Mean 

98.3% 

0.021 

0.032 

0.087 

11186 

St.  Dev. 

0.0204 

0.0120 

0.0155 

0.0302 

6851.3 
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3.2.4  Ordnance.  Five  different  ordnance  types  were  used:  40mm,  60mm,  81mm,  105mm,  and 
155mm.  Each  type  was  buried  at  a  specific  depth:  6,  12,  18,  24,  and  30  inches.  The  ordnance 
was  placed  in  both  the  horizontal  and  vertical  orientations.  The  Pd  measurements  for  each 
ordnance  type,  depth,  and  orientation  are  presented  in  Table  3.2-2. 


TABLE  3.2-2.  NOVICES  WITH  INSTRUMENTATION  -  Pd  BY  ORDNANCE 

ORIENTATION  AND  TYPE 


Type 

Novices 

40mm 

100.0% 

a 

60mm 

92.0% 

o 

•S 

81mm 

96.7% 

'j- 

o 

HH 

105mm 

100.0% 

155mm 

93.3% 

Total 

96.2% 

40mm 

100.0% 

fmt 

60mm 

100.0% 

81mm 

100.0% 

U 

QJ 

K. 

105mm 

100.0% 

155mm 

100.0% 

Total 

100.0% 

3.2.5  Demographics 

a.  Prior  to  traversing  the  test  grid,  each  participant  executed  a  basic  demographic 
questionnaire. 

b.  The  group  of  novice  participants,  when  comparing  un- instrumented  with  instrumented 
samples,  was  identical  and  contained  the  same  personnel.  Demographics  of  novice  participants 
can  be  found  in  section  2.2.5. 

3.2.6  Hearing.  The  group  of  novice  participants,  when  comparing  un-instrumented  with 
instrumented  samples,  was  identical  and  contained  the  same  personnel.  Hearing  assessment 
results  of  novice  participants  can  be  found  in  section  2.2.6. 

3.2.7  Lane  Orientation  Effects 


a.  The  complete  test  grid  of  33  lanes  was  divided  into  two  groups,  corresponding  to  their 
magnetic  compass  heading. 

b.  Lanes  1  through  20  was  situated  north  to  south,  and  was  23.703  by  1.5  meters 
(35.6  m  ).  There  were  28  ordnance  scattered  throughout  the  20  lanes.  The  performance  data  of 
the  novice  operators  with  instrumentation  for  lanes  1  through  20  are  presented  in  Table  3.2-3. 
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TABLE  3.2-3.  SUMMARY  TABLE  OF  ALL  OPERATORS  WITH  INSTRUMENTATION 

FOR  LANES  1  THROUGH  20 


1 

Operator 

P„ 

FAR, 

1/m2 

DOP 

Lane 

Velocity, 

m/s 

Total 
Time,  s 

Novices 

N-l 

96.4% 

0.024 

0.043 

0.084 

5806 

N-2 

96.4% 

0.014 

0.038 

0.120 

4049 

N-3 

96.4% 

0.018 

0.087 

6489 

N-4 

100.0% 

0.051 

0.051 

0.038 

13696 

N-5 

100.0% 

0.017 

0.017 

0.118 

4243 

Mean 

97.9% 

0.025 

0.090 

6857 

St.  Dev. 

0.0196 

0.0149 

0.0126 

0.0331 

3960.1 

c.  Lanes  21  through  33  was  positioned  east  to  west.  Lanes  21  through  26  were  17.55  by 
1.5  meters  (26.325  m2),  and  lanes  27  through  33  were  25.00  by  1.5  meters  (37.5  m2).  There  were 
32  ordnances  placed  throughout  these  thirteen  lanes.  The  performance  data  without 
instrumentation  for  only  lanes  21  through  33  are  presented  in  Table  3.2-4. 


TABLE  3.2-4.  SUMMARY  TABLE  OF  NOVICE  OPERATORS  WITH 
INSTRUMENTATION  FOR  LANES  21  THOUGH  33 


1 

Operator 

Pc 

FAR, 

1/m2 

DOP 

Lane 

Velocity, 

m/s 

Total 
Time,  s 

Novices 

N-l 

100.0% 

0.014 

0.014 

0.093 

3042 

N-2 

100.0% 

0.012 

0.012 

0.094 

3025 

N-3 

93.8% 

0.012 

0.064 

0.112 

2608 

N-4 

100.0% 

0.029 

0.029 

0.030 

9651 

N-5 

100.0% 

0.012 

0.012 

0.087 

3319 

Mean 

98.8% 

0.016 

0.083 

4329 

St.  Dev. 

0.0280 

0.0073 

0.0221 

0.0310 

2985.9 

3.2.8  Dynamic  Measurements 

a.  Percentage  of  lane  area  covered. 

(1)  Using  the  TMS  data,  the  lateral  distance  between  the  detector  head  and  each  point  on 
a  0.25  meter  grid  within  the  lane  was  calculated  for  each  recorded  coordinate  of  the  detector 
head.  The  number  of  points  on  the  grid  of  which  the  detector  head  came  within  0.25  meters  at 
some  point  during  the  run  was  divided  by  the  total  number  of  points  on  the  grid  and  multiplied 
by  100.  The  result  was  defined  as  the  percent  of  lane  area  covered. 

(2)  Novices  were  observed  to  have  between  95.82  and  97.52  percent  lane  coverage  rates. 
The  average  and  standard  deviation  was  96.80  percent  and  0.0006  coverage  rates,  respectively. 
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b.  Detector  head  height. 


(1)  Using  the  TMS  data,  the  two  sensors’  positions  were  used  to  calculate  a  vector  to 
determine  the  position  of  the  detector  head.  The  ground  altitude  at  the  nearest  surveyed  point 
was  then  subtracted  from  the  altitude  of  the  calculated  detector  head  position.  The  result  was 
defined  as  the  detector  head  height.  The  data  presented  are  considered  to  be  an  average,  as  they 
were  compared  with  the  time-stamped  lane  data. 

(2)  Novices  were  observed  with  average  detector  head  heights  between  6.52  and  9.72 
inches.  The  average  and  standard  deviation  was  7.70  and  1.157  inches,  respectively. 

c.  Detector  head  velocity. 

(1)  Using  the  TMS  data,  the  incremental  distance  traveled  by  the  detector  head  was 
calculated  by  taking  the  calculated  detector  head  position  at  each  instance  and  subtracting  the 
calculated  detector  head  position  at  the  previous  instance.  The  incremental  distance  traveled  was 
then  divided  by  the  time  lapse  (normally  0.1  s).  The  result  was  defined  as  the  detector  head 
velocity. 

(2)  Novices  were  observed  with  detector  head  velocities  between  1.04  and  1.54  m/s.  The 
average  and  standard  deviation  were  1.17  and  0.208  m/s,  respectively. 


TABLE  3.2-5.  SUMMARY  OF  NOVICE’S  DYNAMIC  DATA 


Operator 

Pd 

FAR, 

1/m2 

ROC 

Distance 

Lane 

Velocity, 

m/s 

%  Lane 
Area 
Covered 

Detector 

Head 

Height, 

in. 

Detector 

Head 

Velocity, 

m/s 

Novices 

N-l 

98.33% 

0.0221 

0.0277 

0.09 

95.82% 

8.24 

1.14 

N-2 

98.33% 

0.0133 

0.0213 

0.11 

97.03% 

7.86 

1.04 

N-3 

95.00% 

0.0159 

0.0525 

0.10 

97.03% 

6.61 

1.07 

N-4 

100.00% 

0.0468 

0.0468 

0.04 

97.52% 

6.52 

1.09 

N-5 

100.00% 

0.0150 

0.0150 

0.11 

96.62% 

9.27 

1.54 

Mean 

98.33% 

0.0226 

0.0327 

0.09 

96.80% 

7.70 

1.17 

St.  Dev. 

0.020 

0.014 

0.016 

0.031 

0.006 

1.157 

0.208 
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3.3  DISCUSSION  OF  INSTRUMENTED  DATA 


3.3.1  Performance  Analysis 

a.  A  summary  of  the  performance  data  for  each  expert  and  novice  with  instrumentation  is 
presented  in  Table  3.3-1.  The  Pd  versus  FAR  of  experts  and  novices  is  shown  in  Figure  3.3-1. 


TABLE  3.3-1.  SUMMARY  OF  PERFORMANCE  DATA  OF  EXPERTS  AND  NOVICES 

WITH  INSTRUMENTATION 


Experts 

Operator 

Pd 

FAR, 

1/m2 

DOP 

Lane 

Velocity, 

m/s 

Total 
Time,  s 

E-l 

98.3% 

0.048 

0.051 

0.089 

9304 

E-2 

83.3% 

0.061 

0.177 

0.075 

10602 

E-3 

100.0% 

0.087 

0.087 

0.074 

10661 

E-4 

61.7% 

0.012 

0.384 

0.105 

7574 

E-5 

95.0% 

0.096 

0.109 

0.069 

11626 

Mean 

87.7% 

0.061 

0.161 

0.082 

9953 

St.  Dev. 

0.1593 

0.0333 

0.1325 

0.0147 

1565.4 

Novices 

N-l 

98.3% 

0.020 

0.026 

0.088 

8848 

N-2 

98.3% 

0.013 

0.021 

0.110 

7074 

N-3 

95.0% 

0.016 

0.052 

0.097 

9097 

N-4 

100.0% 

0.042 

0.042 

0.035 

23347 

N-5 

100.0% 

0.015 

0.015 

0.106 

7562 

Mean 

98.3% 

0.021 

0.032 

0.087 

11186 

St.  Dev. 

0.0204 

0.0120 

0.0155 

0.0302 

6851.3 

Figure  3.3-1 .  Pd  versus  FAR  of  experts  and  novices  with  instrumentation. 
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b.  Using  the  chi-square  distribution  at  0.05  significance  level,  the  novice  Pd  with 
instrumentation  was  found  to  be  significantly  greater  than  the  expert  Pd  with  instrumentation. 
Using  the  Mann-Whitney  test  at  the  0.05  significance  level,  no  significant  differences  were 
found  between  the  number  of  false  alarms  and  time,  but  the  DOP  of  the  novices  was  significantly 
less  than  the  experts 

c.  The  comparison  of  average  lane  velocity  with  the  three  performance  measurements 
(Pd,  FAR,  and  DOP)  is  shown  in  Figures  3.3-2  through  3.3-4. 

d.  The  comparison  of  total  time  with  the  three  performance  measurements  is  shown  in 
Figures  3.3-5  through  3.3-7. 


R2  =  0.1245 


Figure  3.3-2.  Experts  and  novices  -  Pd  versus  average  lane  velocity. 


Figure  3.3-3.  Experts  and  novices  -  FAR  versus  average  lane  velocity. 
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Figure  3.3-4.  Experts  and  novices  -  DOP  versus  average  lane  velocity. 


Figure  3.3-5.  Experts  and  novices  -  Pd  versus  total  time. 
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Figure  3.3-6.  Experts  and  novices  -  FAR  versus  total  time. 


Figure  3.3-7.  Experts  and  novices  -  DOP  versus  total  time. 


3.3.2  Ordnance 


a.  The  average  Pa  for  each  ordnance  type/depth  and  orientation  for  all  the  operators 
without  instrumentation  is  presented  in  Table  3.3-2.  All  operators  achieved  100  percent 
detection  rates  on  the  40mm,  both  horizontally  and  vertically  oriented.  Operators  had  the  lowest 
detection  rates  on  the  81mm,  both  horizontally  and  vertically  oriented.  Overall,  vertically 
oriented  ordnance  had  higher  Pa  than  horizontally  oriented  ordnance.  Ordnance  type  and  depths 
were  codependent  (table  3.3-2);  that  is,  since  each  ordnance  type  was  buried  at  a  certain  depth, 
no  infonnation  can  be  deduced  about  ordnance  type  or  depth  separately. 
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Table  3.3-2.  AVERAGE  Pd  OF  ALL  OPERATORS  WITH  INSTRUMENTATION  BY 
ORDNANCE  TYPE/DEPTH  AND  ORIENTATION 


Type 

Depth 

Horizontal 

Vertical 

Total 

40mm 

6  in. 

100.0% 

100.0% 

100.0% 

60mm 

12  in. 

86.0% 

97.1% 

92.5% 

81mm 

18  in. 

83.3% 

85.7% 

84.6% 

105mm 

24  in. 

90.0% 

100.0% 

96.2% 

155mm 

30  in. 

96.7% 

91.4% 

93.8% 

Total 

90.8% 

94.7% 

92.7% 

b.  The  Pd  by  ordnance  orientation  and  type  for  both  novices  and  experts  is  presented  in 
Table  3.3-3.  Novices  had  100  percent  Pd  with  all  ordnance  buried  vertically.  They  also 
performed  equal  to  or  better  than  the  experts  in  every  category  with  the  exception  of  two,  the 
60mm  horizontally  and  vertically. 


TABLE  3.3-3.  EXPERTS  VERSUS  NOVICES  WITH  INSTRUMENTATION  -  Pd  BY 
ORDNANCE  ORIENTATION  AND  TYPE 


Horizontal 

Type 

Novices 

Experts 

Total 

40mm 

100.0% 

100.0% 

100.0% 

60mm 

92.0% 

80.0% 

86.0% 

81mm 

96.7% 

70.0% 

83.3% 

105mm 

100.0% 

80.0% 

90.0% 

155mm 

93.3% 

100.0% 

96.7% 

Total 

96.2% 

85.4% 

90.8% 

Vertical 

40mm 

100.0% 

100.0% 

100.0% 

60mm 

100.0% 

94.3% 

97.1% 

81mm 

100.0% 

71.4% 

85.7% 

105mm 

100.0% 

100.0% 

100.0% 

155mm 

100.0% 

82.9% 

91.4% 

Total 

100.0% 

89.4% 

94.7% 

3.3.3  Audiological  Analysis.  For  all  participants,  hearing  status  was  not  expected  to  have  a 
significant  impact  on  performance  in  the  study  for  three  reasons.  First,  the  signals  from  the 
magnetic  locators  presented  to  the  listeners  were  provided  through  headphones  to  both  ears. 
Therefore,  hearing  loss  in  only  one  ear  should  not  impact  overall  performance,  as  the  better  ear 
would  be  able  to  compensate  for  the  loss  in  the  poorer  ear.  Second,  the  signals  provided  to  the 
listeners  were  of  a  level  high  enough  to  be  above  their  hearing  thresholds  as  shown  in  the  graphs. 
Third,  the  signals  emitted  from  the  magnetic  locators  were  broad  in  their  frequency  spectrum,  so 
that  an  individual  with  a  bilateral  high  frequency  hearing  loss  should  have  been  able  to  use  the 
lower  frequency. 
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3.3.4  Lane  Orientation  Analysis.  A  statistical  analysis  was  done  to  compare  perfonnance  data 
from  lanes  1  through  20  (north-south  lanes)  with  the  data  from  lanes  21  through  33  (east- west 
lanes).  Pd  was  tested  using  the  chi-square  test  at  0.05  significance  level,  while  the  number  of 
false  alanns,  distance  measurement,  and  total  time  were  tested  using  the  Mann- Whitney  test  at 
the  0.05  significance  level.  The  perfonnance  measurements  that  were  significantly  different 
between  experts  and  novices  are  presented  in  Table  3.3-4.  With  instrumentation  for  all  lanes,  the 
novices  had  significantly  better  Pd  and  significantly  shorter  DOP.  Without  instrumentation  for 
all  lanes,  the  novices  and  experts  performed  similarly.  The  significant  difference  between  lanes 
1  through  20  data  and  lanes  21  through  33  data  (total  time  was  not  addressed  in  these 
comparisons  because  the  area  covered  was  not  the  same)  are  presented  in  Table  3.3-5.  For 
experts  and  novices  both  with  and  without  instrumentation,  significantly  more  false  alarms  were 
found  for  lanes  1  through  20  than  lanes  21  through  33.  CFA  are  false  alanns  in  which  four  or 
more  operators  detected  a  false  alarm  within  a  0.25-meter  radius  circle.  For  the  operators  with 
instrumentation  (38  CFA),  89  percent  occurred  in  lanes  1  through  20. 


TABLE  3.3-4.  SUMMARY  OF  SIGNIFICANCE  TESTING  OF  EXPERTS  VERSUS 

NOVICES 


Pda 

#  FAb 

DOPb 

Total  Timeb 

with 

Inst 

Lanes  1-33 

SIG 

— 

SIG 

— 

Lanes  1-20 

SIG 

SIG 

SIG 

— 

Lanes  21-33 

SIG 

- 

SIG 

- 

without 

Inst 

Lanes  1-33 

— 

— 

— 

— 

Lanes  1-20 

— 

— 

SIG 

— 

Lanes  21-33 

- 

- 

- 

- 

aChi-square  distribution,  one-sided  test,  at  0.05  significance  level. 
bMann- Whitney  test,  one-sided  test,  at  0.05  significance  level. 


TABLE  3.3-5.  SUMMARY  OF  SIGNIFICANCE  TESTING  OF  DATA  FROM  LANES  1 
THROUGH  20  VERSUS  LANES  21  THROUGH  33 


Pda 

#FAb 

DOPb 

with  Inst 

Experts 

— 

SIG 

— 

without  Inst 

Novices 

- 

SIG 

- 

aChi-square  distribution,  one-sided  test,  at  0.05  significance  level. 
bMann- Whitney  test,  one-sided  test,  at  0.05  significance  level. 


3.3.5  Stress  Analysis.  See  appendix  M. 
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3.3.6  Dynamic  Data 


a.  The  basis  for  providing  dynamic  tracking  ability  for  the  detector  head  stems  from  a 
desire  to  understand  its  effect  on  operator  performance.  Ultimately,  the  tracking  system 
provided  sufficient  data  to  allow  for  analysis  of  average  detector  head  height,  detector  head 
velocity,  lane  velocity,  and  percent  area  covered. 

b.  A  comparison  of  Pd  versus  the  four  characteristics  is  shown  in  Figures  3.3-8  through 

3.3- 11;  FAR  versus  the  four  characteristics  is  shown  in  Figures  3.3-12  through  3.3-15.  A 
comparison  of  the  distance  DOP  versus  the  four  characteristics  is  shown  in  Figures  3.3-16 
through  3.3-19. 

c.  The  novices’  performance  as  measured  by  Pd  was  closely  grouped  (fig.  3.3-8  through 

3.3- 11),  so  the  dependency  upon  the  performance  characteristics  was  difficult  to  discern. 
However,  for  the  experts,  Pd  performance  suffered  as  the  height  and  velocity  of  the  detector  head 
increased.  In  addition,  though  the  correlation  was  not  as  significant,  Pd  performance  also 
decreased  as  the  lane  velocity  increased. 

d.  The  number  of  false  alarms  by  both  experts  and  novices’  generally  decreased  as  the 
lane  velocity,  detector  head  height,  and  detector  head  velocity  increased  (fig.  3.3-12  through  3.3- 
15).  However,  the  data  are  widely  scattered,  and  the  linear  regression  does  not  closely  match  the 
data,  with  the  exception  of  the  data  for  the  novices  and  lane  velocity. 


Figure  3.3-8.  Experts  and  novices  -  Pd  versus  detector  head  height. 
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Figure  3.3-9.  Experts  and  novices  -  Pd  versus  detector  head  velocity. 


Figure  3.3-10.  Experts  and  novices  -  Pd  versus  lane  velocity. 
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Experts 

Novices 

-Linear  (Novices) 
-Linear  (Experts) 


92.0%  93.0%  94.0%  95.0%  96.0%  97.0%  98.0%  99.0% 

Percent  of  Lane  Area  Covered 


Figure  3.3-1 1.  Experts  and  novices  -  Pd  versus  percent  of  lane  area  covered. 


Figure  3.3-12.  Experts  and  novices  -  FAR  versus  detector  head  height. 
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Figure  3.3-13.  Experts  and  novices  -  FAR  versus  detector  head  velocity. 


Figure  3.3-14.  Experts  and  novices  -  FAR  versus  lane  velocity. 
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Figure  3.3-15.  Experts  and  novices  -  FAR  versus  percent  of  lane  area  covered. 


Figure  3.3-16.  Experts  and  novices  -  distance  ROC  versus  detector  head  height. 
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Figure  3.3-17.  Experts  and  novices  -  distance  ROC  versus  detector  head  velocity. 


Figure  3.3-18.  Experts  and  novices  -  distance  ROC  versus  lane  velocity. 
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Figure  3.3-19.  Experts  and  novices  -  distance  ROC  versus  percent  of  lane  area  covered. 


e.  Once  the  two  performance  measurements  were  combined  into  the  distance  ROC, 
differences  between  the  experts  and  novices  appeared.  The  novices  performed  better  as  the 
detector  head  height  increased,  while  the  experts’  performance  deteriorated  as  the  detector  head 
height  increased  (fig.  3.3-16).  This  should  be  true  only  for  the  novices  up  to  a  critical  detector 
head  height  value,  after  which  perfonnance  should  decrease  because  of  reduced  received  signal 
strength.  A  similar  result  is  shown  in  Figures  3.3-17  through  3.3-18,  but  the  coefficients  of 
determination  (R2  values)  do  not  indicate  a  close  fit  for  the  linear  regressions,  with  the  exception 
of  the  line  for  the  experts  and  detector  head  velocity  (fig.  3.3-17).  The  closer  the  R2  values  are  to 
1.00,  the  greater  the  correlation  of  the  x  and  y  axis  data. 

f.  Both  the  novices  and  the  experts  covered  the  lane  area  fairly  equally  with  little 
variation.  This  translated  into  no  significant  correlation  between  that  characteristic  and  Pd,  FAR, 
and  distance  ROC  (fig.  3.3-11,  3.3-15,  and  3.3-19).  The  performance  measurements  and 
characteristics  for  each  operator  and  the  groups  are  presented  in  Table  3.3-6. 
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TABLE  3.3-6.  SUMMARY  OF  DATA  FOR  EXPERTS  AND  NOVICES 


Experts 

Operator 

Pd 

FAR, 

1/m2 

ROC 

Distance 

Lane 

Velocity, 

m/s 

%  Lane 
Area 
Covered 

Detector 

Head 

Height, 

in. 

Detector 

Head 

Velocity, 

m/s 

E-l 

98.33% 

0.0477 

0.0506 

0.09 

97.13% 

7.09 

1.38 

E-2 

83.33% 

0.0610 

0.1775 

0.07 

92.82% 

8.88 

1.74 

E-3 

100.00% 

0.0875 

0.0875 

0.07 

97.26% 

7.10 

1.73 

E-4 

61.67% 

0.0124 

0.3835 

0.10 

98.36% 

9.79 

2.62 

E-5 

95.00% 

0.0963 

0.1085 

0.06 

96.95% 

9.56 

1.72 

Mean 

87.67% 

0.0610 

0.1615 

0.08 

96.50% 

8.48 

1.84 

St.  Dev. 

0.159 

0.034 

0.132 

0.016 

0.021 

1.311 

0.463 

Novices 

N-l 

98.33% 

0.0221 

0.0277 

0.09 

95.82% 

8.24 

1.14 

N-2 

98.33% 

0.0133 

0.0213 

0.11 

97.03% 

7.86 

1.04 

N-3 

95.00% 

0.0159 

0.0525 

0.10 

97.03% 

6.61 

1.07 

N-4 

100.00% 

0.0468 

0.0468 

0.04 

97.52% 

6.52 

1.09 

N-5 

100.00% 

0.0150 

0.0150 

0.11 

96.62% 

9.27 

1.54 

Mean 

98.33% 

0.0226 

0.0327 

0.09 

96.80% 

7.70 

1.17 

St.  Dev. 

0.020 

0.014 

0.016 

0.031 

0.006 

1.157 

0.208 

g.  All  results  indicate  that  the  position  and  speed  of  the  Schonstedt  detector  head  impact 
the  performance  measurements.  To  investigate  this  further,  the  detector  head  height  and  velocity 
data  were  plotted  against  the  performance  measurements  without  the  novice,  experts,  and  groups 
classifications.  Since  the  range  of  mean  heights  and  velocities  of  the  best  and  worst  groups 
indicated  the  curves  may  be  parabolic,  either  a  linear  or  parabolic  regression  was  inserted  to  fit 
the  data  as  best  as  possible.  The  results  are  shown  in  Figures  3.3-20  through  3.3-25. 

h.  As  shown  in  Figure  3.3-22,  the  relationship  between  the  detector  head  height  and  the 
distance  ROC  data  may  be  approximated  by  a  parabolic  curve,  with  the  better  distance  ROC 
measurements  achieved  in  the  7-  to  8-inch  range.  When  the  distance  ROC  measurement  is 
broken  into  its  individual  parts  (fig.  3.3-20),  the  detector  head  height  correlates  to  the  Pd 
measurement  more  than  the  FAR  measurement,  as  shown  in  Figure  3.3-21.  This  suggests  that 
operators  may  improve  their  Pd  by  maintaining  the  Schonstedt  detector  head  between  7  and  8 
inches  off  the  ground;  however,  this  will  not  necessarily  improve  their  FAR. 

i.  With  the  high  coefficient  of  determination,  the  parabolic  curve  in  Figure  3.3-25  fits  the 
data  well.  The  curve  suggests  that  the  best  performance  can  be  achieved  by  swinging  the 
Schonstedt  so  that  the  detector  head  travels  at  a  velocity  between  1  and  1.25  m/s.  The  curves  in 
Figures  3.3-23  through  3.3-25  imply  that  the  correlation  was  related  more  to  Pd  performance  than 
FAR  performance,  but  both  may  be  optimized  by  maintaining  a  velocity  in  this  range. 
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Figure  3.3-20.  Pd  versus  detector  head  height,  all  participants. 
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Figure  3.3-2 1 .  FAR  versus  detector  head  height,  all  participants. 
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Figure  3.3-22.  Distance  ROC  versus  detector  head  height,  all  participants. 


Figure  3.3-23.  Pd  versus  detector  head  velocity,  all  participants. 


3.3-16 


Figure  3.3-24.  FAR  versus  detector  head  velocity,  all  participants. 


Figure  3.3-25.  Distance  ROC  versus  detector  head  velocity,  all  participants. 
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3.3.7  Observations 


a.  Overall,  the  novices  had  higher  Pd  and  lower  FAR  within  the  instrumented  subgroup. 
Consequently,  it  was  not  expected  that  DOPs  for  the  novice’s  were  also  considerable  lower 
(factor  of  five  times),  on  average,  than  those  for  the  experts.  The  novices  had  a  very  low  Pd 
standard  deviation  (0.0204)  compared  with  the  experts  (0.1593).  The  Pd  versus  FAR  figure 
strongly  depicts  novices  in  a  tight  upper  left-hand  corner  fonnation,  while  experts  were  loosely 
grouped  more  to  the  right  and  lower  on  the  Pd  axis.  This  matches  the  results  from  the  DOP 
calculations  presented  in  Table  3.3-6.  The  novices  also  exhibited  FARs  generally  one -third 
lower  than  those  of  the  experts. 

b.  Two  trends  can  be  seen  from  the  expert  versus  novice  comparison  plots 
(fig.  3.3-14  and  3.3-18):  the  quicker  the  novices  traversed  the  lanes,  the  fewer  false  alarms  they 
indicated,  and  they  also  had  a  better  overall  performance  (low  DOP).  Also,  the  experts  and 
novices  both  had  similar  lane  velocities  (0.08  and  0.09  m/s).  It  is  interesting  to  observe  that  Pd 
suffered  only  as  the  experts  increased  lane  velocities,  whereas  the  novices  tended  to  not  be 
affected  by  lane  velocity. 

c.  When  reviewing  perfonnance  characteristics,  some  observations  were  clear  with  regard 
to  the  time  domain  presented  in  Table  3.3-6.  Consistent  with  average  lane  velocity,  the  experts 
had  less  variation  among  themselves  then  the  novices  for  total  time;  however,  the  experts’  time 
was  slightly  higher  than  that  of  the  novices.  In  addition,  the  standard  deviation  of  experts  Pd  was 
quite  high  compared  with  that  of  the  novices  (0.159  versus  0.020). 

d.  Overall,  vertically  oriented  ordnance  had  higher  Pd  than  the  horizontally  oriented 
ordnance.  The  novices  located  100  percent  of  the  targets  with  a  vertical  position.  The  experts 
were  able  to  detect  only  the  155mm  target  in  the  horizontal  orientation  better  than  the  novices. 
Both  novices  and  experts  had  difficulty  locating  the  61mm  and  81mm  mortar  targets.  This  may 
have  been  because  of  their  distinctive  profile  of  the  latter  targets  compared  with  the  40mm, 
105mm,  and  155mm  targets,  which  share  a  simple  projectile  geometry.  It  was  unexpected  that 
both  experts  and  novices  located  (Pd  rates  of  1)  the  40mm  projectile  despite  the  predominantly 
aluminum  alloy  composition,  albeit  this  target  was  positioned  at  a  mere  6-inch  burial  depth. 
Overall,  the  novices  averaged  approximately  10  percent  higher  Pd  rates  than  the  experts. 

e.  Experts  showed  an  overall  lower  average  total  time  of  2.7  hours,  whereas  the  novices 
reported  an  average  total  time  of  3.1  hours.  Novice  4  is  an  outlier  in  this  data  set  and  accounts 
for  the  difference  in  average  total  time.  It  is  assumed  that  if  novice  4  performed  in  a  similar 
manner  to  his  novice  counterparts,  this  average  would  be  lower.  Novice  4’s  time  perfonnance 
also  accounts  for  the  considerably  large  standard  deviation  within  the  novice  data  set  of 
1.9  hours.  This  number  is  considered  significant,  considering  the  expert  standard  deviation  was 
26  minutes. 

f.  The  demographics  of  experts  and  novices  seemed  to  provide  a  wide  range  of 
individuals.  Ages,  races,  and  marital  status  seemed  vary  with  no  conclusive  pattern.  The 
majority  of  the  participants  (9  of  10)  were  male,  with  only  one  female  in  the  expert  group, 
education  levels  seemed  to  be  higher  in  the  novice  group,  with  two  participants  having  obtained 
a  bachelor  degree.  The  experts  all  held  high  school  diplomas,  with  1  or  2  years  of  additional 
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study.  It  is  possible  that  expert  participants  interpreted  their  military  training  as  education,  and 
this  may  account  for  the  differences.  All  experts,  but  only  one  novice,  reported  prior  military 
experience.  Of  that  military  experience,  the  experts  had  an  average  of  11.8  years  of  military, 
4.5  years  of  UXO,  and  2.3  years  of  Schonstedt  specific  experience.  One  novice  reported  1.5 
months  of  UXO  and  Schonstedt  experience;  however,  this  was  assumed  to  be  a  subjective 
answer  as  he  had  graduated  from  the  UXO  program  1.5  months  before  participation  in  this  study. 
In  addition,  this  novice  had  not  worked  on  a  UXO  site  between  graduation  from  the  UXO 
training  program  and  participation  in  this  study.  It  was  determined  that  these  data  would  be  valid 
for  inclusion  for  analysis. 

g.  The  heaviest  expert  was  consistent  with  the  individual  greatest  in  height,  and  the 
lightest  expert  was  the  shortest.  No  physical  outliers  were  observed  in  either  set. 

h.  It  is  interesting  that  most  experts  (four  of  five)  reported  having  excellent  health, 
whereas  all  novices  reported  having  good  health.  All  experts  were  nonsmokers,  and  only  one 
novice  reporting  using  tobacco  products. 

i.  The  dynamic  data  provided  a  first-ever  look  into  detector  motion  characteristics. 
Within  the  expert  and  novice  groups,  the  novices  held,  on  average,  the  detector  head 
approximately  three-quarters  of  an  inch  lower  than  the  experts.  Also,  the  novices  had  detector 
head  velocity  rates  approximately  40  percent  slower  than  the  experts.  Overall,  the  data  revealed 
that  operators  may  improve  their  Pd  by  maintaining  the  Schonstedt  detector  head  between  7  and 
8  inches  off  the  ground;  however,  this  will  not  necessarily  improve  their  FAR.  In  addition,  it 
was  observed  that  best  performance  can  be  achieved  by  swinging  the  Schonstedt  so  that  the 
detector  head  travels  at  a  velocity  between  1  and  1.25  m/s. 
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SECTION  4,  DISCUSSIONS 


4. 1  UN-INSTRUMENTED  VERSUS  INSTRUMENTED  DATA 

4.1.1  Analysis.  A  statistical  analysis  was  done  to  compare  performance  data  with 
instrumentation  and  without  instrumentation.  Pd  was  tested  using  the  chi-square  distribution  at 
0.05  significance  level,  while  the  number  of  false  alarms,  distance  measurement,  and  total  time 
were  tested  using  the  Mann-Whitney  test  at  the  0.05  significance  level.  The  perfonnance 
measurements  that  were  significantly  different  between  operators,  with  and  without 
instrumentation,  are  presented  in  Table  4.1-1.  For  all  lanes,  the  experts  had  a  significantly  higher 
Pd  without  instrumentation,  and  the  novices  had  a  significantly  greater  number  of  false  alarms 
and  greater  DOP  without  instrumentation.  The  differences  between  the  operators  with  versus 
without  instrumentation  for  Pd,  FAR,  DOP,  and  time  are  shown  in  Figures  4.1-1  through  4.1-4. 


TABLE  4.1-1.  SUMMARY  OF  SIGNIFICANCE  TESTING  OF  OPERATORS  WITH 
INSTRUMENTATION  AND  WITHOUT  INSTRUMENTATION 


Pda 

FAR,  b  no. 

DOP, b  no. 

Total  Timeb 

Lanes  1  to  33 

Expert 

SIG 

- 

- 

- 

Novice 

- 

SIG 

SIG 

- 

Lanes  1  to  20 

Expert 

SIG 

- 

- 

- 

Novice 

- 

SIG 

SIG 

- 

Lanes  21  to 

33 

Expert 

SIG 

- 

- 

- 

Novice 

- 

- 

- 

- 

aChi-square  distribution,  one-sided  test,  at  0.05  significance  level. 
bMann- Whitney  test,  one-sided  test,  at  0.05  significance  level. 
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Figure  4.1-1.  Difference  in  Pd  between  operators  without  instrumentation  and  with 

instrumentation. 
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Figure  4.1-2.  Difference  in  FAR  between  operators  without  and  with  instrumentation. 


Figure  4.1-3.  Difference  in  DOP  between  operators  without  and  with  instrumentation. 
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Experts  Operators 


Figure  4.1-4.  Difference  in  total  time  between  operators  without  and  with 

instrumentation. 


4.1.2  Discussion 


a.  This  investigation  allowed  for  two  independent  testing  scenarios  that  included  changes 
to  boundary  conditions  of  basic  experimental  design.  The  un-instrumented  variant  provided  a 
scenario,  fairly  close  to  true  UXO  sweep  conditions,  consistent  to  what  one  would  find  if 
contracted  to  perfonn  such  a  mission.  The  instrumented  variant  provided  a  scenario  with  the 
sole  purpose  of  obtaining  critical  human  motion  data  while  allowing  working  conditions  similar 
to  the  un-instrumented  variant. 

b.  Clear  indicators  of  Pd  differences  are  shown  in  Figure  4.1-1  when  comparing  expert  and 
novice  participants.  Despite  expert  4’s  signs  of  outlier  status,  it  seems  that  data  points  show 
greater  variability  in  the  expert  versus  novice  group.  The  larger  slopes  and  variation  between 
data  points  shows  that  Pd  rates  were  highly  variable  in  the  un-instrumented  versus  instrumented 
set.  This  supports  the  argument  that  the  instrumentation  had  a  greater  effect  on  expert  versus 
novice  participants. 

c.  In  addition,  when  comparing  the  basic  Pd  versus  FAR  figure  of  both  data  sets,  it  is  clear 
that  experts  performed  substantially  less  with  the  instrumentation. 

d.  FARs  were  scattered  within  both  groups  with  no  clear,  distinctive  pattern,  as  shown  in 
Figure  4.1-2. 

e.  The  differences  in  DOP  measurements  (fig.  4.1-3)  showed  greater  variability  among 
the  experts  compared  with  the  novices.  Again,  expert  4  was  the  primary  indicator  of  this 
variability. 
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f.  The  differences  in  total  time  (fig.  4.1-4)  show  that  instrumentation  times  were  greater 
then  without.  It  is  hypothesized  that  this  difference  is  due  to  the  three  main  components  with  the 
added  instrumentation  equipment: 

(1)  Extra  mass  to  the  Schonstedt. 

(2)  Unfamiliar  Schonstedt  response  due  to  change  in  localized  magnetic  field  around 
detector  shaft. 

(3)  Cumulative  stress  originating  from  those  discussed  above. 

g.  Expert  4  is  an  outlier  in  both  the  instrumented  and  un-instrumented  data  sets.  It  is 
interesting  that  this  participant  also  had  high  Pd  rates  in  both  testing  scenarios  and  scored  the 
highest  FAR  in  both  scenarios.  Expert  4  was  noted  by  field  observers  to  be  conservative  in 
pacing  and  slower  compared  with  other  participants  in  the  study.  Based  on  the  observations, 
taking  an  exceptionally  slow  pace  while  executing  a  UXO  sweeping,  may  result  in  high  Pd  rates; 
however,  FAR  will  suffer  as  a  result.  This  observation  is  needed  to  be  further  studied  for 
confirmation. 
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4.2  CONCLUSIONS  AND  FUTURE  STUDY 


4.2.1  General 


a.  Prior  hypotheses  generated  notions  that  experts  would  lead  the  sample  population  in 
performance;  data  collected  within  this  investigation’s  boundary  conditions  proved  differently. 
In  general,  experts  scored  worse  in  two  rudimentary  metrics:  Pd  and  FAR.  This  was  true  in  both 
un-ins trumented  and  instrumented  test  scenarios. 

b.  Target  detection  trends  showed  that  vertically  oriented  ordnance  had  overall  higher  Pd 
rates  than  horizontal  ordnance.  Ordnance  with  geometries  of  cylinders  containing  ogives  also 
presented  higher  Pd  rates.  Participants  had  difficulty  locating  mortar-shaped  targets  in  both  the 
vertical  and  horizontal  orientations.  The  horizontal  orientation  of  mortars  specifically  proved 
difficult  for  all  participants,  whereas  all  other  ordnance  geometries  positioned  horizontally  had 
the  highest  Pd  rates. 

c.  Lane  orientation  analysis  showed  that  in  both  test  scenarios,  a  significantly  higher  FAR 
was  found  in  north-south  lanes  versus  lanes  situated  east-west.  Over  86  percent  of  common  false 
alarms  were  observed  in  the  north-south  lanes.  North-south  lanes  accounted  for  36  percent  of  the 
total  test  grid  area. 

d.  The  capture  of  human  and  detector  motion  provides  perspective  into  maximizing 
operator  perfonnance  via  continual  improvements  regarding  operator  actions  during  UXO 
sweeps.  Within  the  boundary  conditions,  the  investigation  determined  that  maintaining  the 
detector  head  at  a  height  of  7  to  8  inches  above  grade,  and  concurrently  swinging  the  device 
between  1  and  1.25  m/s,  provided  the  highest  probability  for  increased  performance. 

e.  Differences  in  performance  measurements  were  found  between  the  instrumented  and 
un-instrumented  variants.  Expert  participants  were  affected  by  telemetry  equipment  attached  to 
the  detector  and  themselves.  This  could  account  for  the  perfonnance  discrepancy  between  the 
expert  and  novice  data  set  in  the  instrumented  variant.  The  novices,  however,  performed  in  a 
similar  manner  as  the  un-instrumented  variant  and  continued  to  outperfonn  the  experts.  This 
occurrence  could  have  been  due  to  the  standardized  and  recent  training  received  from  the  UXO 
Technician  Level  I  training  curriculum.  Furthermore,  this  could  explain  the  variations  in  expert 
performance,  as  their  backgrounds,  military  service,  UXO  and  Schonstedt  experience,  and  other 
UXO  training  was  diverse  in  nature.  Despite  these  differences,  data  suggest  that  continued 
education  in  what  the  UXO  community  identifies  as  “experts”  may  benefit  overall  performance. 

4.2.2  Anecdotes 

When  considering  all  fonns  of  data  from  this  investigation,  several  caveats  must  be 
mentioned  to  provide  a  review  of  the  results  presented. 

a.  The  un-instrumented  variant  of  this  test  provided  a  realistic  scenario  that  mimicked 
conditions  encountered  by  UXO  technicians  in  the  field.  The  results  provide  the  best  (within  all 
data  generated)  results  in  terms  of  quality  for  the  purposes  of  reviewing  operator  trends  for 
implementation  into  the  training  curriculum. 
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b.  The  instrumented  variant  of  this  investigation  was  the  first  look  into  detector  motion 
trends  in  the  UXO  arena.  While  several  sources  of  error  must  be  included  with  the  data,  it  is 
suggested  that  improvement  of  the  tracking  system  in  a  manner  that  reduces  error  will  result  in 
higher  quality  data.  As  a  tertiary  concern,  the  sample  size  of  five  participants  limited  the  results 
of  an  analysis  based  on  descriptive  statistics.  Observations  included  are  valid  only  for  the 
population  observed  within  the  boundary  conditions.  Although  extrapolating  to  a  larger 
population  of  UXO  technicians  is  theoretically  possible,  this  approach  is  discouraged  until  a 
larger  set  of  participants  is  observed. 

4.2.3  Future  Study 

a.  The  current  data  set  would  benefit  most  from  an  increase  in  overall  population, 
specifically,  one  that  satisfies  minimum  sample  size  requirements.  Therefore,  it  is  suggested  that 
testing  continue  to  confirm  original  data  observations  and  increase  data  quality  in  an  effort  to 
present  logical  and  cogent  UXO  operator  observations.  While  the  dynamic  data  captured  during 
the  initial  phase  of  testing  provided  insight  into  detector  and  human  motion,  it  is  suggested  that 
this  phase  of  testing  be  continued  when  a  technology  is  available  to  track  motion  with  less 
invasive  telemetry.  Sensors  with  limited  mass  and  ferrous  content  would  provide  a  reduced 
amount  of  stress  and  unfamiliar  detector  operation,  increasing  data  value.  Continued  testing 
would  only  encompass  analog  methods  of  data  capturing  with  a  focus  on  two  to  three  parameters 
of  the  initial  investigation. 

b.  It  is  critical  to  identify  how  the  results  will  benefit  the  UXO  community  in  the  future. 
An  international  effort  encompassing  the  current  UXO  detection  process  is  under  way  to 
incorporate  quality  assurance  while  emphasizing  process  efficiency  in  current  system  planning. 
The  process  may  benefit  by  absorbing  certain  characteristics  from  the  continuous  improvement 
of  methodologies.  Direct  application  of  observed  trends  into  training  curricula,  in  an  effort  to 
increase  operator  performance  and  decrease  overall  cost,  would  benefit  the  UXO  community. 
This  investigation  is  the  first  step  in  the  discovery  of  trends  that  show  promise  of  increasing 
overall  “MAG  and  flag”  effectiveness. 
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SECTION  5.  APPENDIXES 


APPENDIX  A.  EXPECTED  RECOVERY  DEPTHS  AND  EMPLACEMENT  DATA 

TABLE  A- 1 .  EXPECTED  RECOVERY  DEPTHS 


Ordnance 

Maximum  calculated 
depth,  m 

Maximum  recovered  depth,  m 
(99%  of  UXO  were  found) 

37-mm  Projectile 

0  to  2.400 

0  to  0.76 

40-mm  Projectile 

0  to  3.600 

0  to  0.66 

57-mm  Projectile 

0  to  1.680 

Oto  0.91 

60-mm  Mortar 

Oto  0.910 

0  to  0.46 

75 -mm  Projectile 

0  to  3.020 

Oto  1.22 

8 1  -mm  Mortar 

0  to  1.650 

0  to  1.06 

105-mm  Projectile 

Oto  5.380 

0  to  0.67 

155-mm  Projectile 

0  to  8.700 

Oto  0.91 

3 -in.  Stokes  Mortar 

Oto  2.100 

Oto  0.91 

6 1  -mm  Mortar 

0  to  0.660 

Oto  0.91 

8 1  -mm  Mortar 

0  to  1.650 

Oto  1.22 

M9  Rifle  Grenade 

0  to  0.076 

0  to  0.61 

35-mm  Rocket 

0  to  0.305 

0  to  0.76 

TABLE  A-2.  EMPLACEMENT  DATA 


Ordnance 

Depth,  m 

Delta  (maximum  recovery),  m 

40-mm  Projectile 

0.152 

0  to  .41 

60-mm  Projectile 

0.305 

0  to  .15 

81 -mm  Projectile 

0.541 

0  to  .60 

105-mm  Projectile 

0.715 

0  to  .06 

155-mm  Projectile 

0.917 

0  to  .  1 5 
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APPENDIX  B.  TEXAS  A&M  (TEEX)  TRAINING  SCHEDULE 


Subject 

Location 

Introduction 

IUTP,  Riverside 

UXO  Environmental  Remediation  Overview 

IUTP,  Riverside 

Mathematics,  Electricity  and  Physics 

IUTP,  Riverside 

Explosives  and  Explosive  Effects 

IUTP,  Riverside 

Fuze  Functioning 

IUTP,  Riverside 

Ordnance  Safety  Precautions 

IUTP,  Riverside 

Ordnance  Identification  (Surface  Ordnance) 

IUTP,  Riverside 

Progress  Test  1 

IUTP,  Riverside 

Ordnance  Identification  (Surface  Ordnance) 

IUTP,  Riverside 

Ordnance  Identification  (Surface  Ordnance)  (ID  Application) 

IUTP,  Riverside 

Ordnance  Identification  (Air  Ordnance)  (ID  Application) 

IUTP,  Riverside 

Ordnance  Identification  (Air  Ordnance)  (ID  Application) 

IUTP,  Riverside 

Ordnance  Identification  (Air  Ordnance) 

IUTP,  Riverside 

Ordnance  Identification  (Chemical  Ordnance)  (ID  Application) 

IUTP,  Riverside 

Ordnance  Identification  (Underwater  Ordnance  and  Pyrotechnics) 

IUTP,  Riverside 

Ordnance  Identification  Application 

Practical  Area 

Progress  Test  2 

IUTP,  Riverside 

Demolition  Materials 

IUTP,  Riverside 

Firing  Systems  (ID  Application) 

IUTP,  Riverside 

Disposal  Procedures  (ID  Application) 

IUTP,  Riverside 

Storage,  Handling,  and  Transportation  of  Explosives 

IUTP,  Riverside 

Nonelectric  Firing  Systems  Application 

Demolition  Range 

Nonelectric  Firing  Systems  Test 

Demolition  Range 

Shock  Tube  Firing  Systems  Application 

Demolition  Range 

Electric  Firing  Systems  Application 

Demolition  Range 

Electric  Firing  Systems  Test 

Demolition  Range 

Detection  Equipment  Application  (ID  Application) 

IUTP,  Riverside 

Detection  Equipment  Application  (ID  Application) 

Practical  Area 

Detection  Equipment  Application  (ID  Application) 

Practical  Area 

Detection  Equipment  Application  (ID  Application) 

Practical  Area 

Professional  Development  and  Industry  Seminar 

IUTP,  Riverside 

Equipment  Inventory  and  Maintenance 

IUTP,  Riverside 

Course  Review  (ID  Application) 

IUTP,  Riverside 

Comprehensive  Test 

IUTP,  Riverside 

Course  Critiques  and  Graduation 

IUTP,  Riverside 

IUTP  =  International  Unexploded  Ordnance  Training  Program. 


TEST  ASSIGNMENT  CHART 


Test  Tvpe  Topics  Covered 


1  Written  Math,  Electricity,  Physics,  Explosives  and  Explosive 

_ Effects  (EEE),  Fuze  Functioning _ 


2  Written  Ordnance  Safety  Precautions,  Ordnance  Identification 


3  |  Practical _ | Nonelectric  Firing  Systems _ 


Practical  Electric  Firing  Systems 


5  |  Written _ |  Comprehensive  Course  Exam _ 
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O 

I 


1  Expert  and 
10  Novice 

Level  I  UXO  Technicians  on 
each  test  apparatus 


( 


* 


Probability  of  Detection  (Pd) 
Probability  of  False  Alarm  Rates  (Pra) 
Background  Alarm  Rates  (BAR) 
Miss  Distance  from  Ordnance 


_ t _ 

Performance  measurements  based  on 
test  apparatus,  target  types,  verbal 
reporting  conditions  with  Pfa, 
completion  time  and  confidence 
ratings 


\  Generation  of  cognitive 
\  model  for  enhanced 
j  UXO  Technician 
>•  /  training  curriculum 


)  „ 

Maximized  UXO 
detection  capability  with 
existing  technology 

APPENDIX  C.  FLOWCHART 


APPENDIX  D.  TESTBED 
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APPENDIX  E.  DATA  ANALYSIS  DETAILS 


Recorded  data  will  be  reviewed  for  any  patterns  of  observable  trends.  Statistical  modeling 
or  regression  calculations  will  be  performed  when  relevant  correlation  situations  are  identified. 

In  general,  operators  will  be  divided  into  distinct  sample  sets,  depending  on  their  UXO 
background.  For  the  purposes  of  this  study,  expert  operators  are  those  individuals  with  prior 
military  experience  and  a  minimum  of  5  years  experience  performing  MAG  and  flag  operations. 
Novice  operators  are  those  individuals  that  are  direct  graduates  of  the  Texas  Extension  Service 
(TEEX)  UXO  Level  I  certification  course.  These  designations  will  also  be  made  in  accordance 
with  experience  related  to  the  Geonics  EM61  and  Schonstedt  detectors. 

The  following  are  descriptions  of  data  reviewed: 


Pd- 


UXO  technicians  are  given  the  task  of  locating  items  buried  under  the  surface  of  the 
ground.  The  results  of  their  actions  can  be  graded  by  asking  a  simple  question:  “Did  the  UXO 
technician  find  subsurface  targets?”  This  can  mathematically  be  described  and  simplified  into  a 
true/false  scenario.  A  probability  of  detection  can  be  calculated  that  will  represent  that  of  the 
UXO  technician  locating  UXO  in  a  given  area.  Equation  1  shows  how  the  probability  of 
detection  or  Pd  is  calculated: 


det 

T 

V  ±  total  J 


Equation  1 


where  tdet  is  the  number  of  targets  detected  per  lane  and  T totai  represents  the  total  number  of 
targets  possible  per  lane. 

Pd  will  be  calculated  with  information  recorded  by  the  TMS  system  and  will  then  be 
compared  with  ground  truth  data.  Each  target  will  have  an  imaginary  1 -meter  safe-halo.  If  the 
UXO  technician  declares  a  target  within  that  halo,  it  counts  as  a  detected  target.  If  multiple 
declarations  are  made  within  a  targets  halo,  only  one  will  count  as  a  detected  target.  The  other 
“excess”  declarations  will  not  count  against  the  Pd  score. 

FAR. 


When  a  UXO  technician  declares  a  target  position  that  lies  outside  the  1 -meter  halo  of  a 
validated  target,  a  false  alarm  is  recorded.  Equation  2  defines  the  false  alarm  rate  as 


FAR  = 


(  F  \ 

1  A 

yArea  y 


Equation  2 
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where  FA  is  the  number  of  false  alanns  recorded  and  “area”  is  the  lane  area  in  nr.  If  an  operator 
declares  multiple  targets  outside  of  the  1 -meter  halo,  each  multiple  target  counts  as  a  recorded 
false  alarm. 


Forward  velocity. 

The  operator’s  average  forward  velocity  can  be  defined  as 

ry  \ 

total  delays  ~ 

-  Equation  3 

Lanelength  J 

where  Ttotai  is  the  time  recorded  from  start  to  finish  of  lane,  taudeiays  is  any  obvious  delays 
incurred  during  the  course  of  the  lane  and  Laneiength  is  the  length  of  the  lane  in  meters. 

Detector  Height. 

Detector  height  can  be  defined  as  the  distance  from  the  lowest  possible  point  on  a  detector 
head  to  the  ground  directly  under  the  detector  when  raised.  This  will  be  captured  over  the  course 
of  each  lane  via  laser  positional  data  and  subtracted  from  ground  topography.  Real-time 
kinematic  surveying  technology  was  used  to  achieve  sub-centimeter  z-axis  accuracy.  Detector 
height  can  therefore  be  defined  as 

II d  =  Detector abs  -  topography  abs  Equation  4 

where  Hd  is  the  detector  height  above  the  ground,  Detectorabs  is  the  absolute  height  of  the 
detector  in  the  3-d  TMS  environment  and  topography abs  is  the  absolute  grade  elevation  recorded. 

Average  detector  Height. 

The  average  detector  height  can  be  defined  as 

Hd  =  -  Equation  5 

^ . samples  J 

where  Hd  is  defined  in  equation  4,  and  samples  is  an  arbitrary  number  of  samples  taken  during 
the  period  investigated.  The  number  of  detector  height  samples  will  be  limited  based  on  the 
resolution  of  the  TMS  laser  tracking  system.  Anticipated  sampling  rate  is  10  Hz. 
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Sweep  rate. 


The  number  of  sweeping  motions  that  an  operator  makes  per  unit  time  is  defined  as  the 
sweep  rate.  A  sweep  is  a  full  range  motion  from  left  to  right,  or  right  to  left.  The  motions  an 
operator  makes  just  before  locating  a  target  does  not  count  toward  this  value.  This  value  is  found 
by  replaying  the  TMS  data  file  and  plotting  detector  motion.  Full  sweep  motions  are  then 
summed  and  recorded.  The  sweep  rate  can  alternately  be  defined  as 


S„  = 


J]  sweeps,, 
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_  'T 

total  L  delays  J  J 


I 


Equation  6 


where  sweepsiane  is  the  number  of  full  sweeps  per  lane  and  the  denominator  is  take  from  equation 
3  for  total  corrected  time  to  complete  the  lane. 


Sweeps  per  meter. 

The  number  of  sweeps  per  meter  is  the  average  sweeps  per  unit  distance.  It  can  be  found 
by 


Sm  = 


Y,  sweeps  Iain 

\  Lone length  J 


Equation  7 


where  sweepsiane  and  Laneiength  were  described  in  equations  6  and  3,  respectively. 


Total  time. 

Total  time  is  the  total  UXO  mag  and-  lag  time  required  for  full  coverage  of  the  test  grid. 


Average  lane  time. 

Average  lane  time  is  a  statistical  average  of  an  operators  lane  times  for  lanes  1  through  33. 
This  can  be  calculated  as: 


Ltotal 


St 


total 


33 


Equation  8 


where  T totai  is  decribed  in  equation  3  and  the  denominator  will  be  the  number  of  lanes  covered  in 
the  test.  In  this  case,  we  have  33  lanes. 


E-3 


Percentage  of  area  covered. 


By  assuming  that  each  detector  will  have  a  radius  of  effective  detection,  we  can  map  the 
detector’s  coverage  of  each  lane  with  the  TMS  tracking  system.  TMS  data  files  are  viewed  in  an 
AutoCAD  format  and  a  detection  halo  is  projected  around  the  given  TMS  detector  head  point. 
The  TMS  file  is  replayed  and  the  %  area  covered  is  calculated  as 


Area 


cowered 


Area 


xlOO  Equation  9 


where  AreaC0Vered  is  the  area  covered  by  the  movement  of  the  detector  and  Area  is  described  in 
equation  2. 
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Figure  G-l .  40mm  projectile  vertical  orientation. 


Figure  G-3.  Nine  bucket  calibration  area. 


Figure  G-4.  Bucket  calibration  test  in  progress. 


G-2 


I!  ^  IW^eTS 

^  ^>£  ^cMts  *'s 


j  t 

Figure  G-5.  Field  notes  of  bucket/test  pit  layout. 
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CARTRIDGE,  40MM,  TP,  M7S1 


DYE 


Figure  H-l.  40mm,  TP,  M781  Projectile  with  cartridge  case. 


CARTRIDGE,  60MM  TP,  M49  SERIES 


NOTE:  FIN  EXTENSION  HOT  INCLUDED  OH  THE  M49A2  AND  M49A3 


Figure  H-2.  60mm  TP,  M49  Mortar. 
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Figure  H-3.  8 1mm  M82 1  Mortar  with  propelling  charge,  zone  4. 
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Figure  H-4.  105mm  M60  HD  Projectile. 
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Figure  H-5.  155mm  Howitzer  Projectile. 
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Scope 

Identification 

Scientific  Research  Corporation  (SRC),  under  Contract  DAAH01-00-C-A107  Technical 
Direction  Orders  (TDOs)  0010  and  0018  of  the  US  Army  Threat  Systems  Management  Office 
(TSMO)  program,  was  contracted  by  the  TSMO  to  develop  the  Threat  Minefield  System  (TMS). 
This  System  Users  Manual  (SUM)  describes  in  sufficient  detail  the  provided  interfaces  and 
procedures  necessary  for  the  operation  of  the  Phase  II  (version  2.0)  implementation  of  the  TMS. 
This  document  serves  as  CDRL  A031  of  TDOs  0010  and  0018  and  as  partial  satisfaction  of  the 
TMS  task  order  requirements.  This  users  manual  pertains  to  the  following  computer  software 
configuration  items  applications: 

Operator  Client  (OpCli)  version  1.1.0 

Threat  Minefield  System  Master  (TMS  Master)  version  2.1.0 

TMS  Data  Communications/Digital  Signal  Processing  (TMS  DataComms)  version  2.1.0 
Mine  Interactive  Simulation  Program  (MISP)  version  2.1.0 
Countermine  Test  Management  System  version  2001.503 
Countermine  Test  Management  System  Host  (CTMS  Host)  version  1.0.0 
3DiWorkbench  version  1 .0. 1405 
System  Overview 

TSMO  is  an  office  under  the  Project  Manager  -  Instrumentation,  Targets,  and  Threat 
Simulators  (PM-ITTS)  within  the  Program  Executive  Office  for  Simulations,  Training,  and 
Instrumentation  (PEO  STRI).  TSMO  is  developing  a  Distributed  Interactive  Simulation/High 
Level  Architecture  (DIS/HLA)  compatible  threat  node  that  will  provide  distributed  users  with 
validated  operator-in-the-loop  (OITL)  simulations  of  current  and  emerging  threat  systems. 
Providing  OITL  and  hardware-in-the-loop  radar  simulators  through  the  threat  representations 
available  in  the  threat  node  reduces  the  costs  associated  with  testing  and  training  in  threat 
environments  compared  to  tactical  combat  training  and  operations  at  open-air  ranges. 

The  TMS  system  facilitates  demining  testing  and  training  in  either  real  or  virtual 
environments.  The  system  provides  resources  necessary  to  function  as  a  test  bed  for  demining 
instrumentation  development.  Additionally,  TMS  provides  virtual  mines  for  inclusion  into  HLA 
exercises.  This  manual  applies  to  the  entire  TMS  system. 

This  system  represents  Phase  II  of  the  TMS  effort.  Phase  I,  executed  under  a  separate 
contract,  facilitated  the  initial  Evaluator  Workstation  architecture  on  a  single  personal  computer 
(PC)  running  the  Microsoft  (MS)  Windows  2000  operating  system  (OS).  In  Phase  II,  additional 
subsystems  were  added  to  accommodate  necessary  position  measurement  hardware,  mine 
detector  instrumentation,  mine  detector  operator  instrumentation,  virtual  exercises,  and  an 
extensible  hardware  and  software  framework  to  support  multiple  simultaneous  exercises. 
Software  configuration  management  is  performed  for  the  US  Government  by  SRC  using  the  MS 
configuration  management  tool  Visual  SourceSafe. 

TSMO  sponsored  the  original  Phase  I  and  this  Phase  II  implementation  of  TMS.  The 
intended  initial  users  of  the  system  are  the  US  Army  Countermine  (CM)  office  at  Ft.  Belvoir,  VA 
and  the  Engineering  School  within  the  US  Army  Training  and  Doctrine  Command  (TRADOC) 
although  the  system  can  be  conceivably  used  by  other  agencies  performing  similar  tasks  such  as 
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unexploded  ordinance  (UXO)  and  ground  remediation  training.  The  Night  Vision  and 
Electronics  Systems  Directorate  (NVESD)  works  closely  with  the  CM  office  at  Ft.  Belvoir  for 
the  purpose  of  humanitarian  demining  and  other  technology  development  related  activities. 

Phase  I  Software  Contributors 

SRC  was  tasked  by  TSMO  to  develop  a  TMS  that  would  operate  in  conjunction  with 
mine  detection  instrumentation  such  as  the  U.S.  Army  Program  Manager  -  Close  Combat 
Systems  (PM-CCS)  Countermine  (CM)  office  Handheld  Standoff  Mine  Detection  System 
(HSTAMIDS)  and  the  Ground  Standoff  Mine  Detection  System  (GSTAMIDS).  BRTRC  and 
Riggs  Consulting  supported  SRC  through  subcontracts.  BRTRC  authored  the  original 
Countermine  Test  Management  System  (CTMS)  software  primarily  under  NVESD  funding  that 
has  been  used  by  the  CM  community  and  was  adopted  as  part  of  the  TMS  workstation  capability. 
Dr.  Lloyd  Riggs  of  Auburn  University  and  sole  proprietor  of  Riggs  Consulting  aided  in  data 
collection  and  statistical  model  developments. 

Phase  II  Capability  Improvements 

TMS  generates  synthetic  signal  returns  for  the  mine  detection  instrumentation  operator 
based  upon  the  simulated  interaction  between  detectors  and  a  set  of  Virtual  Mine  Models 
(VMMs)  developed  by  SRC.  The  following  four  operational  goals  have  been  realized  by  TMS: 

1.  The  TMS  Survey  function,  in  conjunction  with  the  Geo-location  Wand,  provides 
a  rapid,  cost-effective  method  of  surveying  the  “ground  truth”  of  a  minefield  area. 

2.  TMS  also  provides  an  effective  means  of  indicating  to  the  exercise  evaluator 
whether  a  detector  operator  target  detection  alarm  was  valid  or  not. 

3.  The  TMS  False  Alarm  Assessment  function  aids  an  evaluator  in  the  determination 
of  whether  a  false  target  detection  was  due  to  detector  equipment  or  operator  error. 

4.  The  TMS  CM  Test  Analysis  functions  will  further  reduce  the  difficulty  of 
analyzing  CM  Test  data. 

TMS  can  be  divided  into  functional  units  to  describe  the  modular  approach  to  the  system: 

1 .  Survey  equipment  used  to  survey  a  minefield  to  supply  “ground  truth”  to  the 
workstation  using  either: 

a)  Conventional  surveying  equipment,  which  stores  its  survey  results  in  a 
ground  truth  file 

b)  New  equipment,  known  as  a  Geo-location  Wand,  that  transmits  its  data 

2.  Operator/detector  instrumentation  used  to  collect  and  transmit  position  and  alarm 

data 

3.  A  workstation,  used  by  the  exercise  evaluator,  which  interfaces  with  the  survey 
equipment,  mine  detection  instrumentation  and  its  operator,  and  the  DIS/HLA  network 

4.  A  simulation  program  which  adapts  the  position  and  alarm  data  from  the  detector 
and  its  operator,  computes  and  displays  their  interaction  with  a  real  and/or  virtual  minefield 

5.  A  library  of  VMMs 
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Exclusive  of  the  survey  and  operator  instrumentation,  all  of  the  above  capability  will  be 
resident  within  what  is  described  as  the  workstation.  The  workstation  may  or  may  not  be 
composed  of  multiple  computers  or  platfonns.  The  modular  approach  to  the  system  allows  the 
form  to  match  the  system  requirements  for  a  particular  application.  Below  in  Figure  1-1  is 
shown  the  top-level  functions  of  the  TMS  workstation.  All  modules  under  the  control  of  the 
TMS  Master  executable  (shown  within  the  TMS  Master  box)  are  resident  on  the  workstation. 

This  system  is  intended  to  find  application  at  all  TRADOC  Engineering  School  demining 
training  sites  across  the  United  States  including  Yuma,  Ft.  Leonard  Wood,  Ft.  A.  P.  Hill, 
Aberdeen,  Hawaii  and  others.  The  system  currently  supports  handheld  detection  devices  but  is 
intended  to  be  retrofitted  to  support  vehicular  based  mine  detection  assets  as  well.  The  initial 
handheld  demining  equipment  that  TMS  was  developed  for  is  the  US  Army  issue  AN/PSS-12 
detector  manufactured  by  Scheibel  and  the  newly  developed  Handheld  Standoff  Mine  Detection 
System  (HSTAMIDS)  in  development  by  Cyterra  Corporation  of  Orlando,  FL  under  the 
direction  of  the  CM/NVESD  offices. 
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•  Position,  Locating  and  Tracking  (PLT)  Equipment 

•  Operator  Instrumentation 

The  TMS  Operator  instrumentation  includes  position  sensing  gear,  instrumented  mine  detection 
hardware,  and  the  Operator  Client  (OpCli)  subsystem.  OpCli  includes  data  acquisition, 
signal  injection,  data  communication  and  control  software. 

•  Data  Communications,  HLA  Communications  and  Digital  Signal  Processing 
(DataComms/DSP)  Equipment 

•  Evaluator  Workstation  Equipment 

The  TMS  Evaluator  Workstation  subsystems  consist  of  the  TMS  Master,  and  the  Mine  Interactive 
Simulation  Program  (MISP)  evaluation  tools,  Survey,  the  Countermine  Test  Management 
System  (CTMS)  Host,  and  CTMS  itself. 

•  Shared  Memory  Equipment 
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The  replicated  shared  memory  (RSM)  component  provides  common  data  availability  across  all 
evaluator  workstations  served  by  the  DataComms/DSP  machine.  This  component  is 
generically  called  Runtime  Global  Memory  (RGM).  In  the  mobile  version  of  the  TMS 
system,  RGM  is  facilitated  through  software. 

Document  Overview 

This  SUM  was  developed  in  accordance  with  the  TMS  task  description  and  DI-IPSC- 
81443.  There  are  no  security  or  privacy  considerations  associated  with  its  use. 

The  purpose  of  this  document  is  to  present  the  instruction  for  correct  operation  of  the 
TMS  System  including  the  Hardware  Configuration  Items  (HWCIs)  and  the  TMS  Computer 
Software  Configuration  Items  (CSCIs).  It  describes  the  operations  necessary  to  exploit  the  TMS 
system  capabilities  and  demonstrate  compliance  with  the  program  requirements.  The  SUM  is 
used  as  the  basis  for  describing  the  correct  system  operational  procedures.  It  provides  an 
overview  of  the  system  with  sufficient  background  to  understand  the  functions.  Table  0-1 
provides  an  overview  of  this  document.  This  document  contains  only  unclassified  information. 


Table  0-1.  Overview  of  SUM 


Section 

Title 

Description 

1 

Introduction 

Provides  a  full  identification  of  the  system,  the  software  and  this  document. 

2 

System  Overview 

Provides  a  list  of  documents  referenced  within  this  SUM. 

3 

Software  Summary 

This  section  describes  the  system  and  associated  software  applications, 
inventory,  environment,  organization,  operational  overview,  any  contingencies 
and  assistance  and  problem  reporting  contacts. 

4 

Access  to  the 
Software 

This  section  introduces  the  software  to  the  first  time  user,  describes  session 
initialization  and  session  controls. 

5 

Processing 

Reference  Guide 

This  section  describes  the  system  capabilities,  procedures,  and  any  related 
processing  procedures. 

6 

Notes 

This  section  provides  an  acronym  list. 

7 

Appendices 

This  section  describes  the  Arc  Second  positioning  system  set-up  procedures. 
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Referenced  Documents 

The  following  documents  of  the  exact  issue  shown  are  a  part  of  this  manual  to  the  extent 
specified  herein.  In  case  of  conflict  between  the  documents  referenced  herein  and  the  contents  of 
this  manual,  the  contents  of  this  manual  shall  be  considered  the  superseding  document. 


Table  2-1,  Referenced  Documents 


Ref# 

Identification  # 

Date 

Title 

1 

JT&E  Contract  F08635-97-D- 
0017  Task  Order  0031 

4/11/00 

TMS  Task  Description 

2 

DI-IPSC-81438 

12/5/94 

Software  Test  Plan  (STP)  Data  Item  Description 

3 

JT&E  Contract  F08635-97-D- 
0017  T.O.  0031  (CDRL  B001) 

6/22/00 

Threat  Minefield  System  Software  Requirement 
Specification  (SRS) 

4 

N/A 

6/22/00 

Contract  F08635-97-D-0017  T.O.  0031  Threat 
Minefield  System  Preliminary  Design  Review  (PDR) 

5 

N/A 

10/11/00 

Contract  F08635-97-D-0017  T.O.  0031  Threat 
Minefield  System  Critical  Design  Review  (CDR) 

6 

JT&E  Contract  F08635-97-D- 
0017  T.O.  0031  (CDRL  B002) 

10/18/00 

Threat  Minefield  System  Software  Design  Description 
(SDD) 

7 

Contract  Number  DAAK70- 
92-D-0003  Task  Order  0056 

1/25/98 

Countermine  Test  Management  System  Draft  User’s 
Guide  and  Technical  Report  V2.0 

8 

Contract  Number  DAAK70- 
92-D-0003  Task  Order  0056 

Apr.  2000 

Countermine  Test  Management  System  2000 

9 

Threat  Systems  Management 
Office  Contract  DAAH01-00- 
C-A107/010 

5/24/01 

TMS  Statement  of  Work 

9 

Threat  Systems  Management 
Office  Contract  DAAH01-00- 
C-A 107/0 18 

7/08/02 

TMS  Statement  of  Work 

10 

Threat  Systems  Management 
Office  Contract  DAAH01-00- 
C-A 107/0 10/0 18  (CDRL 

A0 18) 

4/3/03 

Conceptual  Design  Drawings  and  Associated  Lists 

11 

Threat  Systems  Management 
Office  Contract  DAAH01-00- 
C-A 107/0 10/0 18  (CDRL 

A005) 

5/6/03 

Acceptance  Test  Plan  for  the  Threat  Minefield  System 

12 

Threat  Systems  Management 
Office  Contract  DAAH01-00- 
C-A 107/0 18  (CDRL  A009) 

5/15/03 

Acceptance  Test  Procedures  for  the  Threat  Minefield 
System 

13 

Threat  Systems  Management 
Office  Contract  DAAH01-00- 
C-A 107/0 10/0 18  (CDRL 

A010) 

5/31/03 

Threat  Minefield  System  Phase  II  System  Development 
Final  Report 
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Software  Summary 

TMS  may  be  setup  in  multiple  configurations.  One  optional  system  configuration 
intended  for  pennanent  facility  installations  consists  of  a  single  DataComms/DSP  component 
residing  on  a  dedicated  computer  providing  instrumentation  data  in  real-time  to  multiple 
distributed  evaluator  workstations  through  a  RSM  network.  Another  optional  system 
configuration  suitable  for  single  evaluator  use  includes  a  single  computer  on  which  a 
DataComms/DSP  component  is  co-resident  with  the  evaluator  components,  with  which  it 
communicates  through  local  shared  memory.  For  both  system  configurations,  there  are  options 
on  the  setup  of  other  components  as  well.  The  software  components  of  TMS  reside  on  three  or 
more  different  computers,  depending  upon  the  configuration  in  use.  One  or  more  software 
applications  perform  the  processing  required  to  implement  the  capabilities  of  TMS  on  each 
computer.  These  applications  are  listed  and  described  briefly  in  the  following  sections. 

Software  Applications 

The  following  custom  applications  implement  or  initiate  the  processing  required  for 
TMS.  Data  acquisition  and  communication  processing  on  the  field  instrumentation  unit  (FIU)  is 
performed  by  the  Operator  Client  application.  Data  communication  between  evaluator 
workstations  and  the  FIU(s)  as  well  as  any  data  conversion  and  translation  processing  is 
performed  on  the  DataComms/DSP  component  by  the  TMS  DataComms  application.  Operation 
specification,  configuration,  and  execution  at  an  evaluator  workstation  are  performed  by  the 
TMS  Master  application.  VMM  implementation,  operator  performance  evaluation,  and  mine 
detonation  prediction  are  performed  by  the  MISP  at  an  evaluator  workstation.  CTMS  performs 
exercise  scoring  and  area  coverage  analysis.  Data  transfer  between  an  evaluator  workstation  and 
an  external  workstation  executing  CTMS  is  facilitated  by  the  CTMSHost  application. 
Additionally,  several  commercial  applications  perform  some  TMS  functionality,  either  as  a 
stand-alone  application  or  under  the  control  of  one  of  the  custom  TMS  applications. 

Software  Inventory 

The  files  required  to  execute  each  of  the  TMS  software  applications  are  listed  in  the 
following  sections.  A  TMS  software  installation  utility  will  install  all  of  the  custom  components 
necessary  for  a  particular  computer/configuration.  Commercial  components  will  be  installed  by 
either  the  TMS  installation  utility  or  an  installation  utility  provided  by  the  component  developer, 
as  required. 

Operator  Client 

TMS  custom  components 

OpCli.exe  -  the  Operator  Client  application 

ws_dll.dll  -  WinSock  communications  processing 
StartApp.exe  -  process  start-up  application 
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Commercial  components 

3DiWorkbench.exe  -  Arc  Second  control  application 

Conductor.dll  -  Primary  assembly  containing  the  core  objects  that  make  up  a  3Di 
system 

MsgTransport.dll  -  Network  library 
PositionData.dll  -  Position  calculation  algorithms 
MathLibCS  -  Mathematic  and  geometric  calculation  library 
Symantec  PCAnywhere  (multiple  files) 

TMS  DataComms 

TMS  custom  components 

TMSDataComms.exe  -  the  TMS  DataComms  application 
dc_net_io.exe  -  IP  network  communications  processing 
dc_serial_io.exe  -  serial  interface  processing 
dc_proc.dll  -  TMS  DataComms/comm.  processing  shared  memory 
ws_dll.dll  -  WinSock  communications  processing 
serial_buff.dll  -  serial  interface  physical  layer  processing 
rgm_p2.dll  -  Phase  2  shared  memory  API  and  implementation 
datacommshelp.hlp  -  online  help  documentation  file 
datacommshelp.cnt-  online  help  contents  file 

Commercial  components 

DMSO  RTI  HLA  support  package  (multiple  files) 

VR-Link  HLA  support  package  (multiple  files) 

Pnpscr.dll  -  Systran  SCRAMNet  replicated  shared  memory  device  driver 

UART.dll  -  ICP  DAS  low  level  serial  communications  driver  for  Demo 
Unit  controllers  (See  Appendix  B.) 

I7000.dll  -  ICP  DAS  high  level  serial  communications  driver  for  Demo  Unit 
controllers  (See  Appendix  B.) 

Symantec  PCAnywhere  (facility  configuration,  multiple  files) 


TMS  Master 

TMS  custom  components 

TMSMaster.exe  -  the  TMS  Master  application 
Ctmstt.mdb  -  augmented  CTMS  targets  database 
SurveyTmplt2.exe  -  survey  session  data  database  template 
Ws_dll.dll  -  WinSock  communications  processing 

rgm_p2.dll  -  Phase  2  shared  memory  API  and  implementation 

Commercial  components 
StripM.ocx  -  strip  chart  display  ActiveX  control 
Mscomct2.dll  -  date/time  display  ActiveX  control 
Pnpscr.dll  -  Systran  SCRAMNet  replicated  shared  memory  device  driver 
TechSmith  Snaglt  screen  capture  package  (multiple  files) 

Symantec  PCAnywhere  (single  evaluator  configuration,  multiple  files) 
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TMS  MISP 

TMS  custom  components 

TMSMISP.exe  -  the  MISP  application 
Avg_user.mdb  -  average  user  statistics  database 
MineResponse.mdb  -  mine  detector  response  value  database 

rgm_p2.dll  -  Phase  2  shared  memory  API  and  implementation 

Commercial  components 
StripM.ocx  -  strip  chart  display  ActiveX  control 
Pnpscr.dll  -  Systran  SCRAMNet  replicated  shared  memory  device  driver 

CTMS/  CTMSHost 

TMS  custom  components 

TMSCTMSHost  -  the  TMS  CTMS  Host  application 
Ws_dll.dll  -  WinSock  communications  processing 
CTMS  2001.503  package  for  TMS  (multiple  files) 

Commercial  components 

MS  Access  2000  database  management  package  (multiple  files) 

Software  Environment 

All  TMS  software  components  run  in  a  MS  Windows  2000  or  MS  Windows  XP 
environment.  Furthermore,  it  is  recommended  that  TMS  DataComms  run  under  MS  Windows 
2000  Server  when  installed  on  a  dedicated  server-type  computer  as  part  of  the  distributed,  fixed 
site  configuration 

Software  Organization  and  Overview  of  Operation 

The  organization  of  the  TMS  software  components  is  specific  to  the  corresponding 
computer  and  configuration.  All  TMS  custom  components  are  installed  to  a 
\TMS_Home\TMS_Bin  folder  on  the  corresponding  computer.  Commercial  components  are 
installed  to  either  the  same  folder  or  a  folder  specified  by  the  installation  utility  provided  by  the 
developer,  as  required.  The  operation  of  each  component  is  also  specific  to  the  functionality  of 
the  corresponding  computer. 

Field  Instrumentation  Unit 

The  software  on  the  FIU  includes  the  Operator  Client  application  as  well  as  the  Arc 
Second  3DiWorkbench  application.  During  field  exercise  execution,  no  direct  operator 
interaction  occurs  with  the  executing  software.  Rather,  the  FIU  is  configured  so  that  the  StartApp 
application  launches  on  system  power-up,  then  starts  each  of  the  required  individual  processes. 
The  3DiWorkbench  application  functions  as  a  data  server,  providing  position  data  to  the 
Operator  Client  application,  which  sends  the  position  data,  along  with  acquired  mine  detector 
output,  to  the  DataComms/DSP  component.  During  the  calibration  of  the  Arc  Second  PLT 
system,  the  user  will  interact  with  the  3DiWorkbench  application  via  a  remote  system  control 
session.  Refer  to  Appendix  A  for  further  information  on  the  Arc  Second  calibration  process. 
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DataComms/DSP 

The  TMS  DataComms  application  perfonns  the  processing  for  the  DataComms/DSP  component. 
The  user  interacts  with  TMS  DataComms  to  configure  communications  interfaces  to  FIUs  and 
assign  field  entity  types  to  each  communications  interface.  The  user  may  also  monitor  system 
status  infonnation  pertaining  to  inter-component  communications  using  various  facilities  of  TMS 
DataComms.  If  no  changes  to  the  communications  interface  and  entity  assignment  configurations 
are  required,  no  interaction  with  TMS  DataComms  may  occur  during  normal  system  operation. 
In  the  facility  configuration,  TMS  DataComms  resides  on  the  dedicated  DataComms/DSP 
computer.  In  the  single  evaluator  configuration,  TMS  DataComms  resides  on  the  single  evaluator 
workstation.  In  this  configuration,  the  TMS  Master  application  will  launch  the  TMS  DataComms 
application  if  it  is  not  already  running. 

Evaluator  Workstation 

The  TMS  Master  application  and  MISP  together  perform  the  processing  required  for  the 
operations  of  the  Evaluator  workstation.  The  TMS  Master  application  is  the  primary  focus  of 
user  interaction  for  normal  system  operation.  It  performs  the  processing  required  for  setup  and 
execution  for  live  field  and  HLA  exercises  as  well  as  exercise  playback  and  survey  operations.  It 
also  exchanges  data  and  status  messages  with  the  CTMS  workstation,  if  CTMS  is  in  use  during 
exercise  execution.  During  live,  HLA,  or  playback  exercise  execution,  MISP  processes  position 
data  against  exercise  ground  truth  data  and  VMM  data  to  predict  detonations,  detector  response 
to  virtual  mines,  and  operator  performance. 

CTMS  Workstation 

The  CTMS  workstation  is  included  to  provide  a  dedicated  environment  for  CTMS  real¬ 
time  processing  during  an  exercise  due  to  the  high  level  of  processor  activity  required  to  perform 
the  CTMS  display  and  analysis  function.  The  use  of  CTMS  during  an  exercise,  either  live  or 
playback,  is  optional  and  is  specified  by  the  evaluator  during  exercise  setup  at  the  Evaluator 
workstation.  The  CTMS  Host  application  facilitates  the  transfer  of  exercise  data  and  status 
messages  between  CTMS  and  the  Evaluator  workstation.  The  user  interacts  to  some  degree  with 
both  applications,  generally  in  response  to  prompts  triggered  by  status  messages  from  the 
Evaluator  workstation.  The  CTMS  Host  application  will  launch  CTMS  as  necessary.  The  CTMS 
Host  application  also  provides  access  to  the  CTMS  Reports  Manager. 

Contingencies  and  Alternate  States  and  Modes  of  Operation 

This  paragraph  has  been  tailored  out,  as  it  is  not  applicable. 

Security  and  Privacy 

TMS  does  not  contain  or  generate  or  provide  any  special  provisions  for  operating  on 
classified  data.  Also,  TMS  has  no  functional  requirement  for  operating  within  a  secure 
environment. 

Assistance  and  Problem  Reporting 

For  assistance  with  the  operation  and  configuration  of  TMS  or  to  report  a  problem 
encountered  while  using  TMS,  contact: 

Prime  System  Developer: 

Scientific  Research  Corporation 
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Bill  Brothers,  Program  Manager/Senior  Systems  Engineer 
(256)  428-9222 
bbrothers@scires.com 

Government  Technical  Representative: 

TSMO  (AMSTI-ITTS-SSC) 

John  Vanderwilt 

(256)  876-9656,  ext.  222  (commercial) 

476-9656  (DSN) 

John.  V  anderwilt@tsmo  .redstone .  anny .  mil 


Access  To  the  Software 

First  Time  User  of  the  Software 

Each  of  the  TMS  applications  involving  user  interaction  adheres  to  the  familiar  MS 
Windows  user  interface  paradigm.  All  user  controls  are  laid  out  and  arranged  in  the  conventional 
manner.  For  TMS  DataComms,  the  first  steps  for  a  new  user  are  to  create  one  or  more 
communications  interfaces  and  associate  those  interfaces  with  field  entities  of  the  appropriate 
type.  For  TMS  Master,  the  first  steps  for  a  new  user  are  to  configure  and  execute  an  operation  of 
the  appropriate  type.  For  CTMS  Host,  the  new  user  will  follow  the  prompts  resulting  from  status 
messages  received  from  the  Evaluator  workstation.  Refer  to  Section  5  for  more  details  on  the 
specific  user  interactions  with  each  application. 

Equipment  Familiarization 

TMS  Workstations 

All  of  the  TMS  workstation  applications  run  on  standard  IBM  PC-style  computers.  For 
the  facility  configuration,  each  workstation,  including  the  dedicated  DataComms/DSP  computer, 
must  include  a  Systran  SCRAMNet  RSM  board.  Also  in  this  configuration,  each  workstation 
must  include  2  network  interface  controllers  (Ethernet  ports.)  It  is  recommended  that  the 
dedicated  DataComms/DSP  computer  be  a  server-type  PC  with  significant  processing  capability 
to  better  handle  the  network  communications  and  signal  processing  calculations. 

FIU  Processor 

The  TMS  FIU  processor  is  custom-built  to  meet  the  requirements  of  TMS.  It  contains  a 
fully  functional  embedded  Intel  Pentium-based  single  board  computer  running  the  MS  XP 
Professional  operating  system.  Wireless  Ethernet  and  an  Analog-to-Digital  converter  (ADC)  are 
provided  via  Personal  Computer  Memory  Card  International  Association  (PCMCIA)  form  factor. 
The  FIU  also  contains  four  Arc  Second  position  calculation  engine  (PCE)  cards  and  the 
interconnections  required  between  the  embedded  computer  and  the  PCEs.  Figure  4-1  shows  the 
front  panel  of  the  TMS  FIU. 
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Figure  4-1.  FIU  Front  Panel 


The  FIU  is  water  resistant  to  lm.  All  connectors  on  the  front  panel  are  water  resistant. 
The  RF  IN  connector  is  a  water  resistant  SMA  connector  and  the  remaining  connectors  are  auto¬ 
locking  cylindrical  connectors  produced  by  Fischer  Connector,  Inc.  The  front  panel  Fischer 
connectors  and  their  mates  on  the  provided  cabling  are  color-coded.  In  addition  to  being  color- 
coded,  each  port  type  (Power,  Sensor,  Video,  etc.)  has  a  different  number  of  pins  to  prevent 
accidental  connection  mistakes.  The  connectors,  however,  are  not  keyed  differently  so  some 
care  must  still  be  exercised  to  not  force  connectors.  If  the  connectors  are  forced,  significant 
damage  may  occur. 


FIU  Input/Output  Connection  Ports 

Power 

Power  is  supplied  to  the  FIU  though  the  PWR  IN  port.  This  is  a  6-pin  Fischer  connector 
that  is  color-coded  red.  The  mating  cable  provides  two  connectors  for  connecting  to 
rechargeable  batteries.  Either  one  or  two  batteries  may  be  connected.  Additionally,  the  two 
battery  connectors  are  diode-separated  inside  the  FIU,  allowing  for  “hot  swapping”  of  the 
batteries.  Each  15V,  11  Ah  battery  will  provide  at  least  5  hours  of  service.  Each  battery  has  a 
charge  indicator  on  top  to  determine  approximate  percentage  charge  remaining.  Connecting  a 
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fully  charged  battery  to  the  FIU  with  a  depleted  battery  will  generally  only  draw  current  from  the 
fully  charged  battery. 

The  power  and  reset  buttons  are  both  water  resistant.  The  power  button,  labeled  PWR, 
will  be  illuminated  red  while  power  is  applied  to  the  FIU  and  it  is  turned  on.  Pressing  the  power 
button  while  the  FIU  is  on  has  no  effect.  Pressing  the  reset  button  at  any  point  will  cause  the 
embedded  computer  to  perform  a  hard  shutdown,  and  power  to  the  FIU  to  be  turned  off. 
Operation  of  the  reset  button  is  not  standard,  as  the  power  button  must  be  pressed  to  restore 
power  to  the  FIU.  “Graceful”  shutdowns  are  accomplished  remotely.  However,  typically,  there 
are  no  consequences  from  a  hard  shutdown.  Shutdowns  of  any  type  should  not  be  perfonned 
during  an  exercise. 


PS2  -  Keyboard/Mouse 

The  keyboard/mouse  port  allows  a  PS/2  keyboard  and/or  a  PS/2  mouse  to  be  connected  to 
the  embedded  computer.  This  connector  is  a  12-pin  Fischer  connector  labeled  KB/M  and  color- 
coded  green.  The  mating  cable  provides  two  standard  PS/2  connectors.  Also  provided  is  a 
Twiddler  handheld  keyboard/mouse  input  device  (also  called  a  chording  device).  During  nonnal 
operations,  the  keyboard  connector  of  the  Twiddler  (or  replacement  handheld  keyboard  device  if 
desired)  will  be  connected  to  the  keyboard  PS/2  connector  to  allow  the  operator  to  input  alarm 
information.  Normally,  the  mouse  connector  will  not  be  used  because  it  is  not  necessary  for 
operations  in  the  current  configuration.  During  configuration,  or  other  operations  that  require 
comprehensive  access  to  the  embedded  computer,  the  keyboard  and  mouse  connectors  may  be 
used  to  provide  input  from  standard  PS/2  input  devices.  PS2  connections  are  not  plug  and  play 


and  therefore  must  be  present  upon  FIU  power  application  for  PS2  device  recognition  by  the  OS. 
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Video 


The  Video  port  is  a  19-pin  Fischer  connector  that  is  color-coded  blue.  The  mating  cable 
provides  a  standard  15-pin  video  graphics  array  (VGA)  connector.  During  normal  operations, 
the  FIU  is  headless  and  the  video  connector  will  not  be  used.  It  provides  access  to  the  embedded 
computer  video  port  and  will  be  used  for  configuration  and  other  operations  where  a  video 
monitor  is  required. 

Ethernet 

The  Ethernet  port  is  a  5-pin  Fischer  connector  that  is  color-coded  gray.  This  port  provides 
10/100  Ethemet-connectivity  to  the  embedded  computer.  The  mating  cable  provides  a  standard 
RJ-45  connector.  During  normal  operations,  there  will  be  no  connection  made  to  this  port. 

Sensors 

The  FIU  has  four  laser  PLT  system  sensor  ports.  Each  provides  connectivity  to  one  of 
the  internal  PCE  boards.  The  sensor  ports  are  4-pin  Fischer  connectors  color-coded  yellow. 
The  four  ports  are  labeled  LEFT,  RIGHT,  UPPER,  and  LOWER  and  are  intended  to  be 
connected  to  specific  sensors  on  the  operator  and  the  mine  detector.  The  LEFT  port  must  be 
connected  to  the  sensor  on  the  operator’s  left  foot  and  the  RIGHT  port  to  the  sensor  on  his  right 
foot.  The  UPPER  port  should  be  connected  to  the  upper  sensor  on  the  mine  detector  shaft  and 
the  LOWER  port  should  be  connected  to  the  lower  sensor  on  the  mine  detector  shaft.  The 
Sensor  port  mating  cables  mate  to  the  sensor  ports  on  one  side  and  the  detector  housings  on  the 
other  side.  All  Sensor  port  mounting  cables  are  identical,  so  care  must  be  taken  to  attach  the 
correct  sensor  to  each  port. 

Universal  Serial  Bus  (USB) 
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The  USB  port  provides  access  to  the  embedded  computer  USB  interface.  The  USB  port 
is  a  10-pin  Fischer  connector  color-coded  white.  The  mating  cable  provides  two  standard  USB 
connectors.  During  operation,  one  connector  is  allocated  for  the  optional  camera  attached  to  the 
mine  detector  shaft  (a  USB  extension  cable  has  been  provided  to  give  the  operator  full  range  of 
motion).  The  second  USB  connector  is  used  for  a  USB-to-serial  (this  extra  port  is  necessary 
when  RS-232  data  is  being  collected  using  the  Minelab  F3  mine  detector). 

DAQ 

The  Data  Acquisition  port,  labeled  DAQ,  is  a  10-pin  Fischer  connector  color-coded 
white.  This  port  provides  connections  to  the  FIU’s  data  acquisition  ADC  subsystem.  The  mating 
cable  provides  a  standard  9-pin  D  connector.  There  are  four  single  ended  analog  input  channels 
provided.  The  input  range  for  these  channels  is  +10V  to  -10V.  There  is  also  a  single  digital-to- 
analog  converter  output  channel.  This  channel  provides  output  from  +10V  to  -10V  and  can  be 
used  to  inject  signals  into  a  properly  instrumented  mine  detector.  During  TMS  operations,  this 
port  is  used  to  collect  data  from  and  provide  signals  to  an  instrumented  AN/PSS-12.  The 
instrumented  AN/PSS-12  has  a  9-pin  D  connector  that  the  other  end  of  the  DAQ  mating  cable 
plugs  into. 

RF/Wireless  Ethernet 

The  RF  port  is  an  SMA  female  connector.  To  facilitate  802.11b  communications,  an 
antenna  must  be  connected  to  this  port.  The  provided  802.1  lb  antenna  may  be  attached  directly 
to  this  connector  for  short-range  communications  (~l  00m).  For  longer-range  communications 
(up  to  1  mile),  a  provided  4-foot  mast  may  be  attached  using  the  provided  clamps.  A  cable  with 
SMA  male  ends  on  both  sides  is  used  to  connect  the  RF  port  to  the  bottom  of  the  antenna  mast. 
The  provided  antenna  is  then  connected  to  the  top  of  the  mast.  This  raises  the  antenna  above  the 
operator  so  that  the  operator’s  body  does  not  shadow  the  802.1  lb  signals  (attaching  the  antenna 
mast  is  only  necessary  if  802. 1  lb  connectivity  cannot  be  achieved  without  it). 


Audio 

The  Audio  port  is  a  7-pin  Fischer  connector  color-coded  black.  It  provides  access  to  the 
audio  output  of  the  embedded  computer.  This  port  is  currently  unused.  In  the  future,  the  Audio 
port  may  be  used  to  provide  audio  or  other  feedback  to  the  operator. 
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Integrating  the  FIU,  the  Handheld  Mine  Detector  and  the  Operator 


This  section  lists  the  necessary  equipment  and  the  steps  required  to  outfit  a  handheld 
mine  detector  operator  with  an  FIU  prior  to  conducting  an  exercise.  The  AN/PSS-12  is  cited  as 
an  example  mine  detector. 

Equipment  List 

•  TMS  FIU 

•  TMS  FIU  mounting  bracket,  shoulder  straps,  and  belt 

•  Ultralife  15V  lithium  ion  battery 

•  Battery  pouch 

•  802.1  lb  antenna 

•  2  Arc  Second  sensors  with  foot  mounts 

•  2  Arc  Second  sensors  with  mine  detector  shaft  mounts 

•  USB  camera 

•  Instrumented  AN/PSS-12 

•  Twiddler  handheld  input  device 

•  Associated  cabling 


Integration  Steps 

•  Mount  the  FIU  to  its  mounting  bracket  and  attach  the  belt  and  shoulder  straps.  Put  the 
battery  in  the  battery  pouch  and  attach  the  battery  pouch  to  the  belt. 

•  Attach  the  802. 1  lb  antenna  to  the  RF  port  of  the  FIU. 

•  Attach  the  battery  cable  to  the  battery. 

•  Put  the  FIU  on  the  operator  and  adjust  the  belt  and  shoulder  straps  until  the  processor  is 
secure  and  the  operator  is  comfortable.  The  FIU  weight  should  rest  on  the  hips  of  the 
operator  and  not  on  his  shoulders.  The  shoulder  straps  are  provided  for  stability  and 
should  not  be  weight  bearing. 

•  Attach  the  two  Arc  Second  sensors  to  the  shaft  of  the  AN/PSS-12.  Also  attach  the  USB 
camera  to  the  mine  detector  shaft  between  the  two  sensors. 

•  Attach  an  Arc  Second  sensor  to  each  of  the  operator’s  feet  using  the  foot  mounts. 

•  Place  the  AN/PSS-12  electronics  over  the  operator’s  shoulder  such  that  the  electronics 
are  on  the  opposite  side  of  the  operator  from  the  hand  holding  the  AN/PSS-12  during 
operation. 

•  Using  the  provided  cables,  attach  each  sensor  to  the  appropriate  sensor  port  on  the  FIU. 
Take  care  with  the  routing  of  the  cables  to  avoid  impairing  the  operator’s  range  of  motion 
with  the  detector  or  with  his  feet. 

•  Attach  the  USB  camera  to  the  FIU  USB  port  using  the  provided  USB  adapter  cable. 
Follow  the  same  cable  routing  as  was  used  for  the  detector  shaft  sensor  cables. 

•  Attach  the  DAQ  port  to  the  AN/PSS-12  using  the  provided  cable. 
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•  Attach  the  Twiddler  to  the  KB/M  port  using  the  provided  cable.  Only  the  keyboard 
connector  of  the  Twiddler  should  be  used.  The  operator  should  attach  the  Twiddler  to  the 
same  wrist  as  the  arm  swinging  the  mine  detector.  In  this  fashion,  the  other  hand  is  free 
to  adjust  the  AN/PSS-12  sensitivity  control  and  press  the  appropriate  buttons  on  the 
Twiddler  for  alann  and  clutter  enunciations. 

•  Insert  the  battery  into  the  provided  FIU  mounting  belt  pouch.  Attach  the  battery  cable  to 
the  battery  and  the  FIU  PWR  IN  port.  Press  the  PWR  button  to  turn  on  the  FIU. 

Access  Control 

No  access  control  beyond  the  standard  MS  Windows  user  login  process  is  provided  or 
required  for  any  TMS  computer  or  application.  Normal  FIU  operation  is  headless  and  requires 
no  user  interaction  beyond  the  indication  of  alarms  during  an  exercise. 

Installation  and  Setup 

The  procedures  required  for  the  installation  and  setup  of  TMS  components  are  dependent 
upon  the  specific  component  and  the  desired  overall  system  configuration.  The  following 
sections  address  these  types  of  installation  and  setup  issues  for  all  TMS  components. 

DataComms/DSP  Software  Installation  and  Setup 

In  the  distributed,  facility  configuration,  the  DataComms/DSP  component  resides  on  a 
separate  computer  that  also  includes  the  shared  system-wide  Mass  Storage  archive  volume.  This 
configuration  uses  a  Systran  SCRAMNet  RSM  board  to  implement  system  RGM.  The 
installation  of  the  RSM  board  and  its  support  software  is  the  first  step  in  the  setup  of  the 
distributed  configuration  of  DataComms/DSP.  The  next  step  for  the  distributed  configuration  (or 
the  first  step  for  the  mobile  single  evaluator  configuration)  is  the  installation  of  the  TMS 
software,  and  optionally,  the  third  party  HLA  interface  support  software.  Finally,  the  network 
interface  configuration  parameters  must  be  set  and  networking  support  software  loaded.  All  of 
this  software  is  provided  and  installation  procedures  are  described  in  the  following  sections. 

Replicated  Shared  Memory  Installation  and  Setup 

Note:  this  procedure  is  required  only  for  the  distributed,  facility  installation. 

•  Install  the  RSM  board  in  the  computer  following  the  instructions  in  the  Systran  hardware 
reference  manual.  When  appropriate,  connect  the  fiber  optic  cables  between  the  board  in 
the  DataComms/DSP  computer  and  those  in  all  workstation  computers  following  the 
instructions  in  the  same  manual.  (Note:  all  cables  must  be  connected  for  the  RSM 
network  to  function  properly.) 

•  Install  the  RSM  support  software  for  the  appropriate  version  of  Windows  following  the 
instructions  in  the  Systran  programmer’s  reference  manual. 

•  Launch  the  WINInst  application  from  the  SCRAMNet  program  group  in  the  Programs 
submenu  of  the  Windows  Start  menu.  Click  the  Edit  button  on  the  initial  dialog.  In  the 
board  configuration  dialog,  enter  1  as  the  Node  ID.  Leave  the  default  values  for  all  other 
settings.  Click  OK  to  save  the  new  value. 

•  Exit  WINInst.  Restart  Windows. 
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DataComms/DSP  TMS  Software  Installation 


The  specific  installation  of  the  TMS  software  for  the  DataComms/DSP  component  is 
dependent  upon  the  system  configuration  and  the  requirement  for  HLA  interface  support.  HLA 
interface  support  is  optional  because  it  is  implemented  using  third  party  components  under  a  per- 
platfonn  license. 

Distributed,  Facility  Configuration 

•  From  the  TMS  Software  Installation  CD,  if  HLA  support  is  required,  run  the 
Facility  DC  HLA  Setup  installation  utility.  If  HLA  support  is  not  required,  run  the 
Facility  DC  NoHLA  Setup  installation  utility. 

•  The  installation  utility  will  create  a  folder  named  ‘Archive  Storage’  at  the  root  of  the  G: 
drive.  Under  Windows,  designate  this  folder  as  Shared. 

•  The  installation  utility  will  create  a  folder  named  ‘TMS  DB’  within  a  ‘TMS  Home’ 
folder  that  is  at  the  root  of  the  F:  drive.  Under  Windows,  designate  this  folder  as  Shared. 

Single  Evaluator  Configuration 

The  software  for  the  DataComms/DSP  component  in  the  single  evaluator  configuration  is 
installed  together  with  all  other  TMS  software.  See  Section  4. 1.3.2. 3.2. 

HLA  Interface  Software  Installation 

The  HLA  interface  implemented  within  TMS  DataComms/DSP  incorporates  2  third  party 
components,  the  runtime  infrastructure  (RTI)  and  VR-Link.  Each  component  is  installed 
separately  as  described  in  the  following  sections. 

RTI  Installation 

The  RTI  is  government-owned  software  that  facilitates  the  communication  between  the 
federates  in  an  HLA  federation.  TMS  uses  the  Defense  Modeling  and  Simulation  Organization 
(DMSO)  1.3NG  version  4  RTI  (RTI1.3NGv4).  The  RTI  was  originally  downloaded  from  the 
DMSO  web  page;  however,  DMSO  no  longer  provides/supports  the  RTI.  The  necessary  version 
is  included  on  the  TMS  Software  Installation  compact  disc  (CD).  Execute  the  RTI-1.3NGv4- 
Win2000.exe  installation  utility  and  install  the  RTI  software  to  the  folder  C:\Program 
Files\DMSO\RTI1.3NG-v4/Win2000-vc6.  Associated  with  the  RTI  is  a  configuration  file  used 
by  DataComms/DSP  named  RTI. rid.  This  file  contains  settings  specific  to  the  particular 
installation  of  DataComms/DSP  and  the  network  on  which  DataComms/DSP  is  an  HLA 
federate.  This  file  is  installed  as  part  of  the  TMS  software  installation  in  the  ‘TMS  Bin’  folder. 
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VR-Link  Installation 


VR-Link  is  a  product  of  MAK  Technologies  that  provides  a  programming  interface  to  the 
RTI.  Each  installation  of  VR-Link  requires  a  separate  runtime  license.  Therefore,  VR-Link  must 
be  installed  from  an  installation  package  purchased  for  a  specific  installation.  Refer  to  the 
instructions  included  with  the  VR-Link  installation  package.  VR-Link  uses  the  FLEXlrn  runtime 
license  manager.  FLEXlrn  is  included  in  the  VR-Link  installation.  Before  you  can  run  an 
application  that  employs  VR-Link,  a  valid  license  file  must  be  obtained  from  MAK 
Technologies.  The  license  file  is  keyed  to  the  host  ID  of  the  TMS  DataComms/DSP  computer. 
The  license  file  must  reside  in  the  ‘TMS  Bin’  folder  within  the  ‘TMSHome’  folder.  It  will  be 
named  ‘host.lic,’  where  ‘host’  is  the  host  name  of  the  computer.  The  TMS  DataComms 
application  automatically  executes  the  license  manager  by  invoking  the  TMSRUNLM  batch  file 
located  in  the  ‘TMS  Bin’  folder.  This  file  should  be  edited  to  reference  the  correct  license  file. 
In  addition  to  the  license  file,  the  environment  variable,  MAKLMGRD  LICENSE  FILE,  must 
be  defined  on  the  TMS  DataComms/DSP  computer  as  @host  where  ‘host’  is  the  host  name  of 
the  computer.  Refer  to  the  VR-Link  users  manual  for  more  information  on  the  installation  and 
usage  of  the  runtime  components  of  VR-Link. 

Network  Interface  Parameter  Settings  and  Support  Software  Setup 

DataComms/DSP  uses  Ethernet  networks  to  exchange  data  with  FIUs  (as  IEEE  802.11b 
Wireless  Ethernet,)  to  connect  to  HLA  simulations,  and  to  share  locally  resident  data  files  and 
exercise  archive  files  with  distributed  workstations.  In  addition,  third  party  support  software 
must  be  installed  along  with  the  Wireless  Ethernet  hardware.  The  exact  configuration  of  the 
Ethernet  network  interfaces  used  by  DataComms/DSP  is  dependent  upon  the  requirements  of  the 
specific  installation. 

Field  Instrumentation  Network  Interface  Parameter  Settings  and  Support  Software 

The  field  instrumentation  network  is  implemented  as  Wireless  Ethernet.  If 
DataComms/DSP  is  installed  on  a  desktop  or  workstation  computer,  an  external  access  point  is 
used  to  connect  to  the  field  instrumentation  units.  For  increased  network  capacity,  the  access 
point  may  actually  be  a  central  outdoor  router  (COR)  that  supports  2  simultaneous  channels.  If 
necessary  to  support  a  large  number  of  field  units  simultaneously,  a  second  COR  could  be  added. 
DataComms/DSP  is  connected  to  the  access  point  through  a  built-in  network  interface.  The  IP 
address  of  this  network  interface  on  DataComms/DSP  must  be  consistent  with  the  IP  address 
assigned  to  the  access  point.  Support  software  supplied  with  the  access  point  is  used  to  configure 
the  IP  address  and  network  operating  parameters  of  the  access  point.  One  network  parameter  will 
indicate  that  the  network  is  operating  through  an  access  point.  If  DataComms/DSP  is  installed  on 
a  notebook  computer,  a  Wireless  Ethernet  interface  card  may  be  installed  directly  in  a  PC-Card 
slot.  A  different  set  of  support  software  is  used  to  configure  the  interface  card  and  network 
operating  parameters.  One  network  parameter  will  indicate  that  the  network  is  operating  among 
peers.  (The  IP  addresses  assigned  to  field  instrumentation  units,  as  well  as  the  network  operating 
parameters,  must  also  be  consistent  with  those  assigned  to  the  instrumentation  network  interface 
of  the  DataComms/DSP  computer  and  the  access  point.  These  settings  are  assigned  locally  to 
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each  instrumentation  unit.)  Refer  to  the  user  manual  of  the  appropriate  support  application  for 
details  on  configuring  the  network  interface  and  operating  parameters,  including  IEEE  802.11b 
channel  assignment. 

In  the  distributed  facility  configuration,  each  distributed  evaluator  workstation  is  also 
connected  to  the  field  instrumentation  network  (using  an  appropriate  network  connection  device, 
such  as  a  hub  or  switch)  in  order  to  receive  video  directly  from  associated  instrumentation  units. 
This  network  connection  is  also  used  to  connect  shared  drives  located  on  DataComms/DSP  in 
which  reside  data  files  and  exercise  archive  files.  Therefore,  the  IP  addresses  assigned  to  the 
corresponding  network  interfaces  on  the  evaluator  workstations  must  be  consistent  with  those  of 
DataComms/DSP  and  the  instrumentation  units.  The  following  address  assignment  scheme  is 
recommended  for  the  instrumentation  network.  This  scheme  requires  that  the  leading  address 
component  (octet  0)  is  150  or  greater. 


Table  4-1.  Recommended  Instrumentation  Network  Address  Assignment 


IP  Address  (Y  >=  150) 

Unit 

Y.X.X.l 

Access  Point/COR  1 

Y.X.X.2 

(Reserved:  Access  Point/COR  2) 

Y.X.X.3 

DataComms/DSP 

Y.X.X.4  -7 

Distributed  Evaluator  Workstation  1-4 

Y.X.X.8  -21 

Field  Instrumentation  Unit  1-14 

Note:  the  current  facility  configuration  of  TMS  provides  for  up  to  4  evaluator  workstations.  This 
number  corresponds  to  the  number  of  802.11b  channels  that  could  be  used  simultaneously 
without  potential  interference  from  frequency  overlap.  Additional  workstations  could  be  added  to 
the  current  configuration  with  minor  modifications  to  some  software  components,  and, 
potentially,  the  use  of  replicated  shared  memory  boards  with  greater  capacity.  Such  an  addition, 
however,  may  require  the  use  of  channel  sharing  among  simultaneously  active  field  units.  This 
could  require  a  trade-offs  in  some  operational  capabilities,  such  as  not  transmitting  video  from  a 
field  unit  sharing  a  channel. 

HLA  Network  Interface  Parameter  Settings 

DataComms/DSP  uses  a  built-in  network  interface  to  connect  to  an  Ethernet  local  area  network 
(LAN)  to  exchange  data  within  an  HLA  simulation.  The  configuration  parameters  for  this 
interface  must  match  those  specified  for  the  LAN  and  for  other  simulation  federates.  In  addition, 
configuration  files  used  to  configure  the  RTI  residing  on  DataComms/DSP  and  other  federates 
may  need  to  be  edited  to  specific  operating  parameters  unique  to  each  federate  network  interface. 
For  instance,  if  DataComms/DSP  uses  2  network  interfaces,  the  IP  address  and  port  number  for 
the  computer  executing  the  RTI  must  be  specified  in  the  ‘RTI. rid’  file  in  the  ‘TMS  Bin’ 
directory  as: 
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(RtiExecutiveEndpoint  X.X.X.X:N) 

where  X.X.X.X  is  the  IP  address  of  the  network  interface  of  the  computer  executing  the  RTI  and 
N  is  the  port  number.  The  information  in  this  file  may  be  modified  for  each  simulation  in  which 
DataComms/DSP  is  a  federate.  (Note:  a  complete  description  of  all  parameters  and  settings 
necessary  to  participate  as  a  federate  in  an  HLA  simulation  is  beyond  the  scope  of  this 
document.) 

Evaluator  Workstation  Software  Installation  and  Setup 

In  the  distributed,  facility  configuration,  the  evaluator  workstation  components  reside  on 
up  to  4  separate  platforms.  This  configuration  uses  Systran  SCRAMNet  RSM  to  implement 
system  RGM  in  hardware.  The  installation  of  the  RSM  board  and  its  support  software  is  the  first 
step  in  the  setup  of  the  distributed  configuration.  The  next  step  for  the  distributed  configuration 
or  the  first  step  for  the  single  evaluator  configuration  is  the  specification  of  the  network  interface 
configuration  parameters.  The  final  step  is  the  installation  of  the  TMS  software  and  third  party 
support  software. 

Replicated  Shared  Memory  (RSM)  Installation  and  Setup 

Refer  to  Section  4. 1.3. 1.1  and  follow  the  same  procedures,  except  that  the  board  Node  ID 
assigned  for  each  evaluator  workstation  should  be  the  workstation  ID  number  +  1  (e.g.,  board 
Node  ID  for  Workstation  1  is  2.) 

Network  Interface  Parameter  Settings 

In  the  distributed,  facility  configuration  each  evaluator  workstation  is  connected  to  the 
field  instrumentation  network  in  order  to  receive  video  directly  from  associated  instrumentation 
units.  This  network  connection  is  also  used  to  connect  shared  drives  located  on  DataComms/DSP 
in  which  reside  data  files  and  exercise  archive  files.  In  both  the  distributed  and  single  evaluator 
configurations,  an  evaluator  workstation  includes  a  separate  Ethernet  network  interface  dedicated 
to  communications  with  the  external  CTMS  workstation.  In  addition,  if  HLA  interface  support  is 
required  in  the  single  evaluator  configuration,  an  HLA  network  interface  must  be  configured  as 
described  in  Section  4. 1.3. 1.4.2. 

Field  Instrumentation  Network  Parameter  Settings 

Refer  to  Section  4. 1.3. 1.4.1  for  infonnation  on  setting  the  field  instrumentation  network 
parameters  for  an  evaluator  workstation  network  interface. 

CTMS  Network  Interface  Parameter  Settings 

An  evaluator  workstation  uses  a  built-in  network  interface  dedicated  to  exchanging  data 
during  an  exercise  with  an  external  computer  executing  CTMS.  As  this  is  a  private,  dedicated 
network,  the  IP  address  of  this  network  interface  should  be  set  as  10.X.X.Z.  Correspondingly,  the 
IP  Address  of  the  associated  network  interface  on  the  external  CTMS  workstation  will  be  set  as 
10.X.X.Z+  1. 
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Evaluator  Workstation  TMS  Software  Installation 


The  specific  installation  of  the  TMS  software  for  the  evaluator  workstation  components  is 
dependent  upon  the  system  configuration  and  the  requirement  for  HLA  interface  support.  HLA 
interface  support  is  optional  because  it  is  implemented  using  third  party  components  under  a  per- 
platfonn  license. 

Distributed,  Facility  Configuration 

The  distributed,  facility  configuration  supports  up  to  4  evaluator  workstations.  Follow 
these  procedures  at  each  workstation. 

•  From  the  TMS  Software  Installation  CD,  run  the  Facility  Eval  Setup  installation  utility. 

•  The  installation  utility  will  create  a  folder  named  ‘TMSGroundTruth’  within  a 
‘TMS  Home’  folder  that  is  at  the  root  of  the  C:  drive.  Under  Windows,  designate  this 
folder  as  Shared. 

•  Launch  Windows  Explorer.  From  the  ‘Tools’  menu  select  the  ‘Map  Network  Drive’  item. 
Set  the  drive  letter  to  ‘Y:’.  Enter  as  the  Folder  ‘\\DCHOST\ARCHIVE_STORAGE’ 
where  ‘DCHOST’  is  the  host  name  of  the  DataComms/DSP  computer.  Check  the 
‘Reconnect  at  logon’  box.  Click  the  ‘OK’  button. 

•  Again  using  Windows  Explorer,  from  the  ‘Tools’  menu  select  the  ‘Map  Network  Drive’ 
item.  Set  the  drive  letter  to  ‘Z:’.  Enter  as  the  Folder  ‘\\DCHOST\TMS_DB’  where 
‘DCHOST’  is  the  host  name  of  the  DataComms/DSP  computer.  Check  the  ‘Reconnect  at 
logon’  box.  Click  the  ‘OK’  button. 

Single  Evaluator  Configuration 

The  software  for  all  TMS  components  is  installed  in  one  operation  for  the  single 
evaluator  configuration.  From  the  TMS  Software  Installation  CD,  if  HLA  support  is  required, 
run  the  Single  Eval  HLA  Setup  installation  utility.  The  HLA  support  software  must  then  be 
installed  following  the  procedures  in  Section  4. 1.3. 1.3.  If  HLA  support  is  not  required,  run  the 
Single  Eval  NoHLA  Setup  installation  utility. 

Evaluator  Workstation  Third  Party  Support  Software  Installation  and  Setup 

One  or  more  third  party  software  packages  may  be  used  on  the  TMS  evaluator 
workstation  to  facilitate  various  system  capabilities.  This  section  addresses  installation  and  setup 
issues  of  these  packages. 

TechSmith  Snaglt 

TechSmith  Corporation’s  Snaglt  is  used  to  capture  the  video  displayed  in  the  video 
session  window  of  TMS  Master  during  a  live  exercise  to  an  ‘.AVI’  video  file.  To  install  Snaglt, 
follow  the  instructions  in  the  corresponding  installation  CD.  After  the  installation  process 
completes  and  Snaglt  is  launched  for  the  first  time,  a  configuration  wizard  allows  the 
specification  of  application  settings.  In  the  ‘Select  Input’  dialog  select  ‘Region.’  In  the  ‘Select 
Output’  dialog  select  ‘Graphics  file.’  In  the  ‘Hotkey’  dialog  accept  the  default  value.  In  the 
‘Preview’  dialog  select  ‘Off.’  On  the  application  main  window,  click  on  the  ‘Video  Capture’  tool 
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button.  Under  the  ‘Input’  menu,  select  ‘Fixed  Region’  and  then  select  the  ‘Properties’  item.  In 
the  ‘Input  Properties’  dialog,  enter  240  as  the  Width  and  200  as  the  Height  (the  dimensions  of  the 
TMS  Master  video  session  video  display  area.)  Under  the  ‘Options’  menu  select  the  ‘Compact 
View’  item.  Exit  Snaglt.  These  settings  will  be  retained  for  subsequent  Snaglt  sessions. 

Symantec  PCAnywhere 

Symantec  PCAnywhere  is  used  to  facilitate  a  remote  session  between  a  TMS  workstation 
and  an  FIU  to  allow  the  user  to  perfonn  certain  configuration  and  setup  operations  local  to  the 
FIU  prior  to  initiating  an  exercise.  The  remote  session  requires  an  active  Wireless  Ethernet 
connection  between  the  TMS  workstation  and  each  FIU.  In  the  facility  configuration, 
PCAnywhere  should  be  installed  on  the  DataComms/DSP  workstation.  In  the  single  evaluator 
configuration,  it  should  be  installed  on  the  Evaluator  workstation.  PCAnywhere  is  distributed  to 
include  installations  for  both  a  “host”  node  and  a  “remote”  node.  The  FIU  installation  should  be 
designated  as  the  “host”  and  the  workstation  installation  as  the  “remote.”  Refer  to  the 
PCAnywhere  documentation  for  detailed  instructions  on  installation,  configuration,  and  use. 

CTMS  Workstation  Software  Installation  and  Setup 

The  external  CTMS  workstation  provides  a  dedicated  computer  for  perfonning  CTMS 
analysis  and  display  processing  during  a  TMS  exercise  without  affecting  the  real-time  processing 
of  the  TMS  evaluation  and  simulation  components.  The  software  installation  and  setup  of  the 
CTMS  workstation  involves  loading  the  CTMS  software,  the  specification  of  the  network 
interface  configuration  parameters,  loading  the  TMS  software,  and,  finally,  loading  third  party 
support  software. 

CTMS  Installation 

Insert  the  TMS  Software  Installation  CD  and  open  the  CTMS  2001.503  Install  folder. 
Launch  the  Setup  application.  Accept  all  default  entries.  (Note:  Only  CTMS  version  2001.503  is 
currently  fully  compatible  with  TMS.) 

Network  Interface  Parameter  Settings 

The  CTMS  workstation  uses  a  built-in  network  interface  dedicated  to  exchanging  data 
during  an  exercise  with  the  TMS  evaluator  workstation.  As  this  is  a  private,  dedicated  network, 
the  IP  address  of  this  network  interface  should  be  set  as  10.X.X.Z.  Correspondingly,  the  IP 
Address  of  the  associated  network  interface  on  the  TMS  evaluator  workstation  will  be  set  as 
10.X.X.Z-  1. 

CTMS  Workstation  TMS  Software  Installation  and  Setup 

The  TMS  CTMSHost  application  facilitates  the  exchange  of  exercise  information 
between  the  TMS  evaluation  and  simulation  components  and  CTMS.  This  section  describes  the 
installation  and  setup  required  for  this  application. 

•  From  the  TMS  Software  Installation  CD  run  the  CTMS  Host  Setup  installation  utility. 
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•  Launch  Windows  Explorer.  From  the  ‘Tools’  menu  select  the  ‘Map  Network  Drive’  item. 
Set  the  drive  letter  to  ‘Z:’.  Enter  as  the  Folder  ‘\\10.X.X.X\TMS_GroundTruth’  where 
‘10.X.X.X’  is  the  IP  address  of  the  CTMS  network  interface  on  the  TMS  evaluator 
workstation  computer.  Check  the  ‘Reconnect  at  logon’  box.  Click  the  ‘OK’  button. 

CTMS  Workstation  Third  Party  Support  Software  Installation  and  Setup 

MS  Access  2000  must  be  loaded  on  the  CTMS  Workstation  in  order  to  run  the  CTMS 
Reports  Manager.  Follow  the  installation  instructions  provided  with  the  Microsoft  Access 
installation  CD.  Accept  all  default  entries  during  installation. 

Field  Instrumentation  Unit  Software  Installation  and  Setup 

Operating  System  Installation 

The  FIU  is  a  headless  system  that  does  not  contain  a  CD-ROM  or  diskette  drive.  There 
are,  however,  pinned  interfaces  that  are  exposed  on  the  FIU’s  NetCard  II  backplane/motherboard 
that  facilitate  CD-ROM  and  diskette  drive  connectivity  (consult  the  NetCard  II  System  Manual 
for  further  information). 

Before  installing  a  Windows  OS,  there  are  some  BIOS  parameters  that  must  be 
configured.  They  are  listed  below  (the  items  listed  are  assuming  a  PhoenixBIOS  4.0  Release 
6.0): 


•  Main  ->  Boot  Feature  (optional) 

Item:  QuickBoot  Mode 
Setting:  Enabled 

Description:  Enables  Quick  Boot,  which  reduces  the  amount  of  time  that  it  takes 
for  the  processor  to  boot 

•  Advanced  ->  PCI  Configuration  ->  PCI/PNP  ISA  IRQ  Resource  Exclusion 

Items:  IRQ  10 
IRQ  11 

Settings:  Reserved 
Reserved 

Description:  Reserves  IRQs  10  and  11  so  that  the  4  RS-232  ports  needed  for  the 
Arc  Second  PCE  cards  will  have  adequate  IRQ  resource  allocation 
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•  Advanced 


Item:  Installed  0/S 
Setting:  Other 

Description:  Specifies  that  installed  operating  system  is  not  Windows  95  or 
Windows  98  (Windows  XP  Professional  is  installed) 

Upon  completion  of  the  BIOS  configuration,  refer  to  the  Windows  “Getting  Started  Manual”  for 
instructions  on  installing  Windows. 

Instrumentation  Software  Installation 

From  a  network  connection,  run  the  InstrumentationPackageSetup  installation  utility. 

Field  Instrumentation  Network  Interface  Parameter  Settings  and  Support  Software 
Refer  to  Section  4. 1.3. 1.4.1. 

Initiating  a  Session 

An  execution  session  for  the  FIU  is  initiated  at  power-on  of  the  unit.  A  session  on  the 
DataComms/DSP  component  in  the  facility  configuration  is  initiated  by  launching  the  TMS 
DataComms  application.  The  user  may  select  to  add  TMS  DataComms  to  the  Startup  program 
group  of  the  Programs  item  of  the  Windows  Start  menu  to  launch  TMS  DataComms  at  power- 
on.  A  session  on  the  Evaluator  workstation  is  initiated  by  launching  the  TMS  Master  application, 
which  in  turn  launches  MISP  when  appropriate.  Also,  in  the  single  evaluator  configuration,  TMS 
Master  will  launch  TMS  DataComms.  In  this  configuration,  TMS  DataComms  may  be  launched 
individually  for  communications  and  entity  configuration  tasks.  A  session  on  the  CTMS 
workstation  is  initiated  by  launching  the  CTMS  Host  application,  which  in  turn  launches  CTMS 
or  the  CTMS  Reports  Manager,  as  appropriate. 

Stopping  and  Suspending  Work 

The  user  may  tenninate  a  session  in  TMS  DataComms,  TMS  Master,  and  CTMS  Host  at 
any  time  using  any  of  the  standard  Windows  methods  for  nonnal  application  shutdown.  Each 
application  will  display  a  prompt  if  it  is  necessary  to  save  any  modifications  and  as  well  as  that 
shutdown  is  the  action  intended  by  the  user.  The  FIU  should  be  sent  a  “Shutdown”  command 
from  TMS  Master  (at  the  conclusion  of  an  exercise)  or  TMS  DataComms  prior  to  power-down 
of  the  system. 
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Processing  Reference  Guide 
Capabilities 

The  capabilities  of  each  of  the  TMS  workstation  applications  are  accessed  using  the 
standard  user  interaction  techniques  of  the  MS  Windows  desktop  environment. 

Conventions 

No  unique  conventions  beyond  those  of  the  MS  Windows  environment  are  included  in 
any  TMS  software  applications. 

Processing  Procedures 

The  following  sections  describe  the  procedures  used  for  operation  of  each  of  the  TMS 
software  applications. 

TMS  DataComms 

The  TMS  DataComms  application  provides  user  access  to  and  display  of  the 
configuration  and  status  data  of  the  TMS  Data  Communications/Digital  Signal  Processing 
(DataComms/DSP)  component.  DataComms/DSP  combines  hardware  and  software  that 
facilitate  the  transfer  of  sensor  data,  position  data,  simulation  data,  and  system  commands 
between  the  TMS  evaluation  and  simulation  components  and  the  TMS  remote  field 
instrumentation  units.  DataComms/DSP  also  provides  any  intermediate  processing  of  the 
exchanged  data  between  the  sender  and  the  user  of  the  data.  In  addition,  DataComms/DSP 
generates  and  stores  archive  files  of  the  data  received  from  field  units  during  a  live  exercise  and 
accesses  these  files  during  playback  exercises  configured  at  the  evaluator  workstations. 
DataComms/DSP  also  receives  and  processes  the  data  sent  by  survey  equipment  in  support  of 
ground  truth  data  generation  and  manual  exercise  scoring  using  the  TMS  Survey  operation. 

DataComms/DSP  exchanges  various  types  and  fonnats  of  data  with  the  TMS  remote  field 
instrumentation  units  through  two  types  of  communication  interfaces.  The  first  is  a  TCP/IP  based 
mechanism  over  Wireless  Ethernet  (IEEE  802.11b).  The  second  is  a  serial  interface  to  RS-232 
compliant  wireless  modem  devices.  DataComms/DSP  also  serves  as  a  gateway  that  allows  the 
TMS  evaluation  and  simulation  components  to  function  as  a  federate  in  HLA  simulations. 
DataComms/DSP  exchanges  HLA  simulation  data  over  an  Ethernet  Local  Area  Network  (LAN) 
interface. 

In  the  distributed  (facility)  configuration,  DataComms/DSP  exchanges  system  data  and 
information  with  the  evaluation  and  simulation  components  using  two  connection  methods. 
System  data  and  operation  control  information  is  exchanged  through  a  system-wide  RGM 
component  accessible  simultaneously  by  DataComms/DSP  and  the  evaluation  and  simulation 
components.  These  exchanges  occur  between  the  system  components  in  real-time  as  an  exercise 
is  configured  and  executed.  In  this  configuration,  RGM  is  implemented  through  a  hardware  RSM 
network.  A  LAN  interface  provides  access  to  files  stored  on  the  Mass  Storage  volume  resident 
on  the  DataComms/DSP  computer  to  the  distributed  components.  In  the  mobile,  single  evaluator 
configuration,  RGM  is  implemented  through  software,  and  all  processes  share  access  to  the  local 
Mass  Storage  volume. 
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Getting  Started 

The  TMS  DataComms  application  (“TMS  DataComms”)  is  intended  to  require  little  to  no 
user  interaction  to  perfonn  normal  operations  in  support  of  workstation  activities.  However, 
some  configuration  parameters  must  be  entered  and  other  activities  performed  before  TMS 
DataComms  is  prepared  to  provide  data  processing  and  transfers  between  workstations  and  field 
units. 

Connections 

Before  powering  on  the  DataComms/DSP  computer,  make  sure  all  necessary  cables  and 
devices  are  connected  properly.  The  RSM  fiber  optic  cables  of  the  DataComms/DSP  computer 
and  all  active  workstation  computers  should  be  connected  in  a  circular,  daisy-chain  fashion.  The 
DataComms/DSP  computer  should  be  connected  from  Network  Interface  1  using  a  standard 
CAT-5  Ethernet  cable  with  RJ-45  connectors  at  each  end  to  the  appropriate  HLA  LAN  interface 
connection.  The  computer  should  be  connected  from  Network  Interface  2  to  the  instrumentation 
network  switch  unit  using  a  standard  CAT-5  Ethernet  cable  with  RJ-45  connectors  at  each  end. 
All  active  evaluator  workstations  should  be  similarly  connected  to  the  switch  unit.  An  Ethernet 
cable  with  RJ-45  connectors  at  each  end  should  connect  the  switch  to  the  Wireless  Ethernet 
(IEEE  802.1  lb)  Central  Outdoor  Router  (COR).  Each  active  interface  card  in  the  COR  should  be 
connected  from  the  card's  antenna  connection  to  the  antenna  splitter.  The  connectors  at  each  end 
of  the  cables  for  this  connection  will  depend  upon  the  type  of  each  device  in  use.  The  antenna 
splitter  is  connected  to  the  antenna's  coaxial  cable.  (An  optional  RF  signal  amplifier  may  be 
inserted  at  the  splitter-antenna  cable  junction.  If  present,  the  amplifier  will  have  an  A/C  power 
connection.)  Also,  any  serial  devices  should  be  connected  to  the  proper  COM  port  on  the 
computer  with  the  appropriate  cabling.  All  external  devices  should  be  powered  on  before  or 
simultaneously  with  the  computer. 

Startup 

TMS  DataComms  may  be  configured  to  start-up  automatically  upon  power-up  of  the 
DataComms/DSP  computer  or  the  user  may  launch  it  as  a  conventional  Windows  application. 
Upon  start  up  of  TMS  DataComms,  the  configuration  data  from  the  last  execution  session  is 
accessed  and  used  to  detennine  the  start  up  configuration.  This  information  specifies 
communications  interfaces  and  the  field  entities  associated  with  the  interfaces,  if  any.  The 
current  configuration  is  displayed  in  the  application's  main  window,  shown  in  Figure  5-1.  After 
this  window  opens,  the  initialization  of  the  current  configuration  will  commence.  Also  at  start¬ 
up,  when  DataComms/DSP  is  configured  to  serve  as  an  HLA  gateway,  TMS  DataComms  will 
launch  an  external  software  license  manager  process  in  a  command  console  window.  This 
process  must  not  be  terminated  while  TMS  DataComms  is  running. 
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Figure  5-1.  TMS  DataComms  application  main  window. 

The  first  time  TMS  DataComms  is  started  after  installation,  the  user  will  be  prompted  to 
identify  the  default  exercise  file  archive  directory  and  to  locate  the  database  file  containing  the 
virtual  mine  model  data.  The  archive  directory  is  the  ‘Archive  Storage’  folder  and  the  database 
file  is  in  the  ‘TMS  DB’  folder  referred  to  in  Section  4. 1.3. 1.2.1.  This  information  is  recorded 
and  used  during  future  execution  sessions.  (Note:  even  when  installed  as  part  of  the  single 
evaluator  configuration,  TMS  DataComms  should  be  launched  directly  to  perform  this 
configuration  step.) 

Initialization 

After  all  the  start  up  configuration  data  is  obtained,  TMS  DataComms  will  commence  the 
initialization  of  the  configured  communications  interfaces.  This  process  involves  allocating  the 
required  local  system  resources  for  each  interface  and  initiating  the  processing  for  each  interface. 
No  communication  with  the  remote  instrumentation  unit  associated  with  an  interface  is  attempted 
or  required  as  part  of  the  initialization  process.  When  an  interface  initializes  successfully,  its 
status  is  set  to  either  Ready  or  Configured,  depending  upon  the  type  of  interface  (see  Interface 
Activity.)  The  initialization  of  a  communications  interface  may  fail  if  any  of  the  required  system 
resources  cannot  be  allocated  to  the  interface.  When  an  interface  initializes  unsuccessfully,  its 
status  is  set  to  Failed.  No  user  action  is  required  to  start  the  initialization  process  nor  can  the  user 
pause  or  terminate  initialization.  Figure  5-2  shows  an  interface  with  a  status  of  Failed. 
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TMS  DataComms 
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RF1  4  -  Survey  Team  2 

5 

Survey 
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H  5  -  Instr.  Unit 

6 

Serial 
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Unit  5  -  Instr.  Unit  interface  failed 


0  26%  ERROR 


Figure  5-2.  Example  of  a  Failed  Interface 

Once  the  initialization  of  all  of  the  configured  communications  interfaces  has  completed, 
TMS  DataComms  will  attempt  to  attach  to  the  RGM  implemented  through  the  RSM  network.  If 
successful,  TMS  DataComms  will  post  to  RGM  its  own  status  and  that  of  all  configured 
interfaces  for  access  by  the  distributed  TMS  components.  The  final  initialization  status 
information  for  all  configured  elements  is  displayed  in  the  appropriate  locations  of  the  main 
window.  Any  problems  that  occur  during  the  initialization  process  are  noted  in  the  System  Event 
Log.  The  System  Event  Log  is  accessible  using  the  ‘Show  Event  Log’  item  of  the  ‘System’ 
menu.  Figure  5-3  shows  the  System  Event  log  window. 


DataComms  Event  Log 


03/05 1 0:1 8:51  -  Unit  2  •  Instr.  Unit  interface  Configured 

03/05 10:19:29  -  Added  Entity  2  -  HSTAMIDS  #2 

03/05 1 0: 1 9: 47  -  Added  I  nterface  3  -  S  urvey 

03/05 1 0:1 9:58  -  Interface  3  -  Survey  T earn  1  started 

03/05 1 0:1 9:57  -  Unit  3  -  Survey  T earn  1  interface  Ready 

03/0510:20:11  -  Added  Interface  4 -Survey 

03/05 10:20:19  -  Interface  4  -  Survey  Team  2  started 

03/05 1 0:20:20  -  Unit  4  -  Survey  T earn  2  interface  Ready 

03/05 10:22:55  -  Added  Interface  5  -  Instr.  Unit 

03/05 10:22:55  -  Interface  5  -  Instr.  Unit  started 

03/05 10:22:56  -  ERROR:  Unit  5  -  Instr.  Unit  interface  failed 


Figure  5-3.  TMS  DataComms  System  Event  Log  window 
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Interface  Configuration 

An  interface  represents  a  communications  channel  between  DataComms/DSP  and 
another  TMS  component.  An  interface  to  an  instrumentation  unit  may  be  created  or  deleted  by 
the  user.  When  this  type  of  interface,  referred  to  as  an  instrumentation  unit  interface,  is  added, 
the  user  must  provide  a  set  of  configuration  parameters.  Once  the  interface  is  created,  the 
configuration  data  may  not  be  modified.  If  any  instrumentation  unit  interfaces  are  added  or 
deleted  by  the  user  during  an  execution  session,  TMS  DataComms  will  prompt  the  user  to  save 
the  configuration  data  for  the  next  execution  session.  Another  type  of  interface  is  created 
automatically  when  a  workstation  is  detected  as  online  through  RGM.  This  interface  is  referred 
to  as  a  "playback"  interface.  It  is  the  mechanism  by  which  DataComms/DSP  provides  data 
through  RGM  from  an  exercise  archive  file  during  exercise  playback  at  a  workstation.  The  user 
cannot  delete  or  provide  any  configuration  parameters  for  a  playback  interface.  In  addition,  a 
playback  interface  does  not  automatically  persist  across  execution  sessions.  Figure  5-4  shows  a 
playback  interface  corresponding  to  an  online  workstation.  When  DataComms/DSP  is 
configured  to  serve  as  an  FILA  simulation  gateway  for  an  evaluation/simulation  component 
functioning  as  a  simulation  federate,  the  corresponding  playback  interface  for  that  workstation 
will  be  reconfigured  automatically  as  an  HLA  interface  to  process  HLA  simulation  data  between 
the  workstation  and  the  simulation  federation. 


Figure  5-4.  Example  of  a  Playback  Interface 
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Instrumentation  Interface  Parameters 

When  the  user  creates  an  interface  to  an  instrumentation  unit,  the  type  of  interface  and  an 
associated  set  of  parameters  must  be  specified.  The  types  of  instrumentation  interfaces 
correspond  to  the  types  of  communications  devices  used  to  facilitate  the  connection  between 
DataComms/DSP  and  the  instrumentation  unit.  The  interface  types  are:  IP/802  for  TCP/IP  over 
Ethernet  (wired  or  wireless);  Serial  for  binary  data  transfers  over  RS-232  compliant  devices;  PI 
Serial  for  data  transfers  over  RS-232  compliant  devices  using  the  ASCII-text  format  devised  for 
TMS  Phase  1;  and  Survey  for  transfers  of  ASCII-text  data  from  survey  equipment  over  RS-232 
compliant  devices.  Two  configuration  parameters  are  common  to  all  instrumentation  interface 
types.  The  first  is  the  Unit  ID.  This  value  identifies  the  instrumentation  unit  with  which  the 
interface  links  DataComms/DSP.  The  second  is  the  Network  ID.  This  value  represents  an 
"address"  used  by  DataComms/DSP  for  transfers  across  the  interface.  Each  of  these  values  must 
be  unique  to  a  single  interface.  Figure  5-5  shows  the  Add  New  Interface  dialog  in  which  the  user 
enters  the  required  configuration  parameters.  The  parameters  specific  to  the  type  of  interface  are 
listed  in  the  following  sections.  The  configuration  parameters  required  for  each  of  the  serial  (RS- 
232)  interface  types  are  the  same. 


Figure  5-5.  Add  New  Interface  dialog 


IP/802  Interface  Parameters 

The  configuration  parameters  specific  to  IP/802  instrumentation  interfaces  are  the  Host 
ID  and  the  Transport  Type.  The  Host  ID  can  be  either  the  IP  (Internet  Protocol)  address  or  the 
host  name  assigned  to  the  instrumentation  unit.  The  IP  address  should  be  used  if  it  is  known.  The 
transport  type  can  be  one  of  UDP  (User  Datagram  Protocol)  or  TCP  (Transmission  Control 
Protocol.)  UDP  should  be  selected  as  the  default. 
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Serial  Interface  Parameters 

The  configuration  parameters  specific  to  serial  instrumentation  interfaces  are  the  COM 
Port  ID  and  the  Baud  Rate.  The  COM  Port  ID  specifies  which  of  the  COM  ports  installed  in  the 
computer  is  to  be  physically  connected  to  the  communications  device  associated  with  the 
interface.  For  the  interface  to  initialize  successfully,  the  COM  port  must  exist  and  be  functional. 
The  Baud  Rate  selection  must  match  the  configuration  of  the  device  connected  to  the  COM  port 
for  communications  to  occur. 

Creating  an  Instrumentation  Interface 

The  user  may  create  a  new  instrumentation  interface  using  either  the  ‘New  Interface’ 
item  of  the  ‘File’  menu,  the  ‘New’  toolbar  button,  or  by  double  clicking  the  ‘Add  Interface’  item 
of  the  main  window's  list.  Each  of  these  methods  will  open  the  Add  New  Interface  dialog,  shown 
in  Figure  5-5,  in  which  the  appropriate  configuration  parameters  described  previously  may  be 
entered.  The  user  may  also  choose  to  immediately  start  the  new  interface  by  selecting  the  ‘Start 
Interface’  check  box  on  this  dialog. 

Interface  Activity 

Interface  activity  includes  any  type  of  communications  over  an  interface  between 
DataComms/DSP  and  the  associated  system  component.  Before  data  transfer  activity  can  occur 
over  certain  types  of  instrumentation  unit  interfaces,  however,  DataComms/DSP  must  first  detect 
the  presence  of  the  instrumentation  unit.  This  occurs  when  the  instrumentation  unit  responds  to  a 
"ping"  message  sent  automatically  by  DataComms/DSP.  Prior  to  receipt  of  the  ping  response,  the 
corresponding  instrumentation  unit  interface  is  assigned  a  status  of  Configured.  Upon  the  receipt 
of  the  ping  response,  the  interface  status  will  be  set  to  Ready.  When  activity  is  detected  on  an 
interface  with  a  status  of  Ready,  the  status  of  the  interface  is  set  as  Active.  An  interface  to  an 
instrumentation  unit  need  not  be  assigned  to  an  exercise  for  activity  to  occur.  Activity  over  a 
playback  interface  occurs  only  during  the  execution  of  a  playback  exercise.  If  the  activity  ceases, 
the  status  of  the  interface  is  set  as  Ready.  After  a  period  of  inactivity,  the  status  of  an 
instrumentation  unit  interface  initialized  with  a  status  of  Ready  will  be  set  back  to  Configured, 
and  DataComms/DSP  will  resume  sending  the  ping  message  to  the  instrumentation  unit. 

Infonnation  on  current  interface  activity  and  status  is  available  to  the  user  in  the  Interface 
Status  dialog,  shown  in  Figure  5-6.  The  user  may  view  this  dialog  by  selecting  an  interface  in  the 
list  of  the  main  window  and  then  either  selecting  the  ‘Open  Interface’  item  of  the  ‘File’  menu  or 
clicking  the  ‘Open’  toolbar  button,  or  by  double  clicking  the  interface  list  item.  If  the  interface 
was  not  started  previously,  a  ‘Start’  button  will  appear  on  this  dialog  to  allow  the  user  to  start  the 
interface.  The  status  of  the  interface  will  be  set  to  Starting  at  that  time.  The  user  may  also 
tenninate  current  and  future  instrumentation  interface  (but  not  playback  interface)  activity  by 
either  clicking  the  ‘Stop’  button  on  the  Interface  Status  window  or  selecting  the  interface  in  the 
main  window's  list  then  clicking  the  ‘Stop  Interfaces’  toolbar  button.  Upon  the  user  performing 
either  of  these  actions,  the  interface  status  is  set  to  Terminated.  The  user  may  not  terminate  an 
interface  that  is  assigned  to  a  live  exercise  configured  at  a  workstation. 
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Figure  5-6.  Interface  Status  dialog 


Deleting  an  Instrumentation  Interface 

The  user  may  delete  an  instrumentation  interface  with  a  status  of  either  Tenninated  or 
Failed.  To  delete  an  interface,  the  user  may  either  click  the  ‘Delete’  button  on  the  Interface 
Status  dialog  (the  ‘Stop’  button  is  replaced  by  the  ‘Delete’  button  for  an  interface  with  a  status  of 
Terminated  or  Failed)  or  select  the  interface  in  the  main  window  list  then  select  the  ‘Delete 
Interface’  item  of  the  ‘File’  menu. 

Field  Entities  and  Instrumentation  Units 

DataComms/DSP  uses  instrumentation  interfaces  to  communicate  with  instrumentation 
units  associated  with  field  entities. 

Field  Entities 

A  field  entity  is  an  object  that  is  tracked  during  a  TMS  exercise.  This  includes  a  platform 
that  carries  a  detector  system,  such  as  an  operator  carrying  a  hand-held  detector  or  a  vehicle 
carrying  a  detector  array;  and  any  supporting  platform(s),  such  as  a  control  vehicle  for  a  remotely 
operated  detector  vehicle,  or  a  team  member  accompanying  a  hand-held  system  operator.  An 
entity  may  have  some  number  of  distinct  components  that  are  individually  tracked,  such  as  the 
operator's  feet  and  the  detector  head  of  a  hand-held  system.  Associated  with  an  entity  is  a  set  of 
characteristic  information,  including  physical  properties  such  as  dimensions  and  weight,  and  the 
type  of  data  collected  from  the  entity  during  an  exercise. 
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Instrumentation  Units 

An  instrumentation  unit  is  a  TMS  component  attached  to  a  field  entity  to  collect  sensor 
data  from  external  sensors  on  the  field  entity  and  pass  the  data  over  a  communications  interface 
to  DataComms/DSP.  For  live  exercises  involving  one  or  more  real  field  entities,  the 
communications  interface  will  typically  be  implemented  using  some  type  of  wireless 
communications  medium,  such  as  Wireless  Ethernet  or  wireless  modems.  The  Test  ID  value 
associated  with  the  configuration  data  for  an  interface  specifies  the  particular  instrumentation 
unit  corresponding  to  the  interface.  When  an  interface  is  initially  created,  it  is  identified  as 
linking  DataComms/DSP  to  an  instrumentation  unit  using  the  Test  ID. 

Assigning  Interfaces  to  Entities 

The  user  must  identify  the  type  of  entity  to  which  an  interface  (and  its  associated 
instrumentation  unit)  links  DataComms/DSP.  The  user  selects  the  ‘Add  Field  Entity’  item  of  the 
‘Field  Entity’  pop-up  list  in  the  toolbar  area  of  the  TMS  DataComms  main  window.  This  opens 
the  Add  Entity  Data  dialog,  shown  in  Figure  5-7.  This  dialog  contains  a  pop-up  list  of  all 
instrumentation  interfaces  that  are  not  currently  assigned  to  entities,  and  another  pop-up  list  that 
contains  a  fixed  set  of  entity  types.  The  user  selects  from  these  lists  an  interface/instrumentation 
unit  ID  and  an  entity  type,  and  then  clicks  the  ‘OK’  button.  The  Interface  Unit  ID  for  the 
interface  is  then  updated  to  indicate  the  specific  entity  assignment.  An  entity  must  be  assigned  to 
an  interface  at  DataComms/DSP  before  it  can  be  selected  at  a  workstation  for  inclusion  in  a  live 
exercise.  The  user  may  also  use  the  Field  Entity  pop-up  list  in  the  toolbar  area  of  the  TMS 
DataComms  main  window  to  delete  a  current  entity  assignment.  To  do  this,  the  user  selects  the 
desired  entity  from  the  Field  Entity  pop-up  list,  and  then  clicks  the  ‘Delete’  button  on  the 
resulting  View  Entity  Data  dialog.  Entities  that  are  currently  assigned  to  a  live  exercise  cannot  be 
deleted.  In  addition,  if  an  instrumentation  interface  associated  with  an  entity  is  deleted,  the  Field 
Entity  list  entry  is  also  deleted. 


Figure  5-7.  Add  Entity  Data  dialog 
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Sending  Commands  to  Instrumentation  Units 

The  user  may  send  Command  messages  to  instrumentation  units  for  a  variety  of  purposes. 
Care  should  be  taken  in  sending  commands  to  an  instrumentation  unit,  as  some  commands  may 
shutdown  or  restart  the  instrumentation  unit.  Under  normal  circumstances,  commands  to 
instrumentation  units  assigned  to  an  exercise  should  not  be  sent  from  DataComms/DSP.  The  user 
clicks  on  the  ‘Send  Command’  button  of  the  toolbar  to  open  the  Send  Command  Message  dialog, 
shown  in  Figure  5-8.  The  user  then  selects  the  instrumentation  unit  and  command  type  using 
pop-up  lists  on  this  dialog.  The  command  is  sent  when  the  user  clicks  the  ‘Send’  button.  The 
instrumentation  unit  sends  a  command  acknowledgement  if  it  successfully  receives  the 
command.  TMS  DataComms  will  indicate  command  send  failure  after  an  appropriate  period  if 
no  acknowledgement  is  received. 


Figure  5-8.  Send  Command  Message  dialog 


Workstations  and  Exercises 

DataComms/DSP  sends  data  received  over  instrumentation  interfaces  from  instrumentation 
units  to  workstations  running  exercises.  It  uses  the  same  interfaces  to  send  simulation  and 
command  data  from  workstations  to  instrumentation  units. 

Workstations 

A  workstation  is  a  computer  functioning  as  a  distributed  client  of  DataComms/DSP  that 
hosts  the  TMS  exercise  control,  evaluation  and  simulation  components.  DataComms/DSP 
provides  to  a  workstation  through  RGM  the  processed  data  resulting  from  DSP  applied  to  raw 
data  received  from  field  entities  through  instrumentation  interfaces.  DataComms/DSP  also  sends 
to  an  instrumentation  unit  simulation  data  accessed  through  RGM  that  was  generated  at  a 
workstation.  A  workstation  provides  an  indication  of  its  presence  to  DataComms/DSP  through 
RGM.  The  DataComms/DSP  user  may  view  the  status  information  for  any  online  workstation  by 
selecting  the  corresponding  entry  from  the  Workstation  pop-up  list  in  the  toolbar  area  of  the 
TMS  DataComms  main  window.  This  will  display  the  Workstation  status  dialog,  shown  in 
Figure  5-9.  When  TMS  DataComms  first  detects  that  a  workstation  is  online,  it  will  create  a 
playback  interface  for  the  workstation.  Upon  creation,  the  status  of  a  playback  interface  is  set  to 
Uninitialized.  If  a  playback  exercise  is  configured  at  the  workstation,  the  playback  interface 
status  is  set  to  Ready.  While  the  playback  exercise  is  in  progress,  the  status  is  Active.  If  the 
workstation  goes  offline,  the  playback  interface  status  is  set  to  Terminated.  If  the  workstation 
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comes  back  online,  the  status  is  again  set  to  Uninitialized.  When  a  workstation  is  configured  to 
participate  in  an  HLA  simulation,  the  corresponding  playback  interface  is  reconfigured  as  an 
HLA  interface. 


Figure  5-9.  Workstation  Status  dialog 


Exercises 

The  data  transferred  over  the  communications  interfaces  associated  with  one  or  more 
instrumentation  units  is  ultimately  processed  within  the  context  of  an  exercise.  The  exercise 
defines  the  entities  of  interest  as  well  as  the  ground  truth  data  (the  geographical  area  of  the 
exercise,  the  boundaries  of  the  test  lane,  and  the  types  and  locations  of  mines  (real  or  virtual)  and 
other  physical  objects  within  that  test  lane,)  the  name  of  an  archive  file,  if  any,  and  a  set  of 
performance  baseline  data  against  which  the  performance  of  the  entities  under  test  will  be 
evaluated.  Exercises  are  configured  and  controlled  at  workstations  running  the  TMS  exercise 
control,  evaluation  and  simulation  components. 

The  raw  data  for  an  exercise  can  be  generated  live  by  field  entities  or  extracted  from  an 
archive  file  from  a  previous  live  exercise  during  playback  mode.  In  either  case,  TMS 
DataComms  processes  the  raw  instrumentation  data  in  real-time  and  posts  the  processed  data  to 
RGM  for  access  by  the  TMS  exercise  control,  evaluation,  and  simulation  components  on  a 
workstation.  In  turn,  the  TMS  simulation  component  may  generate  data  that  is  posted  to  RGM 
and,  in  the  case  of  a  live  exercise,  passed  by  DataComms/DSP  to  the  appropriate  instrumentation 
unit  through  an  instrumentation  interface.  Once  a  workstation  has  defined  an  exercise,  the 
exercise  configuration  and  status  data  is  available  through  RGM  and  may  be  viewed  by  the  user 
of  DataComms/DSP.  The  user  may  view  the  exercise  configuration  data  using  the  Exercise 
Configuration  Data  dialog,  shown  in  Figure  5-10.  This  dialog  may  be  accessed  by  selecting  an 
entry  for  an  exercise  from  the  Exercise  pop-up  list  in  the  toolbar  area  of  the  TMS  DataComms 
main  window.  The  status  of  a  configured  exercise  is  indicated  as  part  of  the  information 
displayed  with  each  interface  assigned  to  the  exercise  in  the  TMS  DataComms  main  window  list. 
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Figure  5-10.  Exercise  Configuration  dialog 

Displaying  DataComms/DSP  System  Information 

Various  displays  containing  configuration  and  status  information  about  DataComms/DSP 
are  accessible  using  the  System  menu.  These  displays  include  the  System  Configuration  display, 
the  System  Event  Log,  the  Available  Ground  Truth  Data  display,  and  the  Check  Access  Point 
utility  display.  In  addition,  panes  in  the  status  bar  at  the  bottom  of  the  main  display  continuously 
indicate  several  status  values  of  interest.  Each  of  these  methods  of  displaying  system  information 
is  described  in  the  following  sections. 

System  Configuration 

The  System  Configuration  display,  shown  in  Figure  5-11,  provides  the  user  with 
information  on  configuration  and  status  of  a  number  of  items.  This  display  is  accessed  using  the 
‘Show  Configuration’  item  of  the  ‘System’  menu. 
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Figure  5-11.  System  Configuration  display 


System  Event  Log 

The  System  Event  Log,  shown  previously  in  Figure  5-3,  provides  the  user  with  a  running 
list  of  indications  and  descriptions  of  events  of  interest  as  they  occur  during  a  TMS  DataComms 
execution  session.  The  Event  Log  display  is  accessed  using  the  Show  Event  Log  of  the  System 
menu.  This  display  maintains  a  list  of  the  most  recent  system  events.  This  display  may  remain 
open  at  any  time  during  an  execution  session.  System  events  that  indicate  operational  errors  will 
be  tagged  with  "ERROR"  and  the  system  status  pane  of  the  status  bar  will  also  indicate  ERROR. 
The  status  pane  will  be  reset  to  OK  after  the  user  displays  the  Event  Log  to  observe  the  error 
explanation. 
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Available  Ground  Truth  Data 

The  Available  Ground  Truth  Data  display  provides  a  list  of  the  ground  truth  data  files 
stored  locally  on  the  Mass  Storage  (archive)  volume  on  the  DataComms/DSP  computer.  By 
default,  this  is  the  set  of  ground  truth  data  with  which  a  user  of  an  evaluation  workstation  will 
define  an  exercise.  If  the  TMS  DataComms  user  selects  a  file  from  this  list,  the  Ground  Truth 
Data  display,  shown  in  Figure  5-12,  will  open  containing  a  list  of  the  mines,  landmarks  and  lane 
boundary  markers  composing  the  ground  truth  data.  The  ground  truth  data  display  is  accessed 
using  the  ‘Available  Ground  Truth  Data’  item  of  the  ‘System’  menu. 


Figure  5-12.  Ground  Truth  Data  display 


Check  Access  Point  Utility 

The  Check  Access  Point  utility  is  accessed  using  the  ‘Check  Access  Point...’  item  of  the 
‘System’  menu.  This  utility  provides  a  means  of  checking  on  the  operational  status  of  a  Wireless 
Ethernet  (IEEE  802.11b)  access  point.  An  access  point  is  basically  a  gateway  onto  the  wireless 
network  for  a  device  with  a  standard  Ethernet  (IEEE  802.3)  port.  A  Central  Outdoor  Router 
(COR)  is  a  type  of  access  point.  Selecting  the  menu  item  will  open  the  Check  Access  Point 
dialog,  shown  in  Figure  5-13.  This  dialog  contains  a  pop-up  list  of  previously  registered  access 
point  IP  addresses,  text  fields  for  entering  a  new  IP  address,  and  buttons  for  adding  a  new 
address,  deleting  a  previously  registered  address,  and  checking  the  status  of  an  access  point 
associated  with  an  address.  If  the  access  point  is  powered  on  and  connected  to  one  of  the 
Ethernet  ports  of  the  DataComms/DSP  computer,  it  will  send  a  reply  to  the  check  operation.  If  it 
is  either  off  or  disconnected,  no  reply  will  be  received. 
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Check  Access  Point 


Access  Point  IP  Address 


No  AP  Addresses  Registered 
|169  1 254  [39  [i" 


Check 


Add 


Figure  5-13.  Check  Access  Point  dialog 


Status  Bar  Panes 

The  status  bar  panes  at  the  lower  right  of  the  main  window  continuously  indicate  values 
for  various  system  status  items.  The  leftmost  pane  indicates  the  number  of  communications 
interfaces  that  are  currently  active  (sending  or  receiving  data.)  The  center  pane  indicates  the 
percentage  of  space  used  on  the  Mass  Storage  (archive)  volume.  The  rightmost  pane  displays 
"ERROR"  if  an  operational  error  occurs  of  which  the  user  should  be  made  aware;  nominally  this 
pane  displays  "OK."  Additional  information  on  system  operational  errors  appears  in  the  system 
Event  Log.  Once  the  user  views  the  Event  Log,  the  contents  of  the  system  status  pane  will  revert 
to  "OK." 

Exiting  TMS  DataComms 

To  exit  TMS  DataComms,  the  user  may  select  any  of  the  standard  Windows  application 
exit  methods,  including  the  ‘Exit’  item  of  the  File  menu  and  the  Close  box  of  the  main  window. 
Exiting  TMS  DataComms  will  also  terminate  all  DataComms/DSP  activity  and  clear  all  data  in 
RGM  posted  by  DataComms/DSP.  After  selecting  an  exit  method,  the  user  is  alerted  if  there  is 
any  interface  activity  or  if  any  exercises  are  currently  configured  at  any  workstations.  If  so,  the 
user  may  elect  to  cancel  the  exit  process.  Also,  the  user  is  prompted  to  save  any  modifications 
made  to  the  current  interface  and/or  entity  configurations.  During  the  TMS  DataComms  exit 
processing,  all  workstations  attached  to  RGM  are  notified  that  DataComms/DSP  is  unavailable, 
meaning  no  live  exercises  may  be  configured  or  executed.  Upon  exiting  TMS  DataComms,  the 
DataComms/DSP  computer  may  be  shut  down. 
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TMS  Master 

The  TMS  Master  application  is  the  evaluator’s  interface  to  the  Threat  Minefield  System. 
The  Master  is  a  conventional  Microsoft  Windows  application  that  provides  the  mechanism  for 
configuring  a  TMS  operation  and  displaying  the  data  specific  to  each  operation.  The  TMS 
operations  are  Exercise,  Playback,  Survey  and  HLA.  The  Master  also  controls  the  execution  of 
MISP  and  communicates  with  the  external  CTMS  workstation. 

Startup 

At  startup,  the  Master  checks  for  an  executing  instance  of  TMS  DataComms  thru  RGM.  If 
TMS  DataComms  is  not  detected,  a  dialog  is  launched  that  allows  the  evaluator  to  make  a 
selection  to  wait  for  TMS  DataComms,  abort  or  startup  in  a  standalone-processing  configuration. 
Optionally,  when  installed  as  part  of  the  single  evaluator  configuration,  the  user  will  be  prompted 
to  start  up  Master  in  Single  Evaluator  mode  and  can  set  this  mode  as  the  default  for  subsequent 
start-ups.  In  this  mode,  Master  will  launch  TMS  DataComms  on  the  evaluator  workstation  if  it  is 
not  currently  executing. 

The  application  main  window,  shown  in  Figure  5-14,  is  displayed  after  the  Master  is 
started.  The  value  displayed  in  the  right-most  pane  of  the  status  bar  is  the  percentage  of  used 
disk  space  on  the  system  mass  storage  device.  The  fourth  pane  defines  the  system’s  operating 
configuration. 


Figure  5-14.  TMS  Master  application  main  window 

Figure  5-15  shows  the  Master’s  menus  at  startup.  The  ‘Exit’  menu  terminates  the  Master 
and,  if  appropriate,  MISP,  and  sends  an  exit  message  to  the  CTMSHost  application  executing  on 
the  external  CTMS  workstation.  The  ‘Operations’  menu  allows  the  evaluator  to  select  the  mode 


1-48 


of  operation  and  the  ‘TMS  Master  System  Log’  menu  item  on  the  ‘View’  menu  opens  a 
modeless  dialog  containing  system  event  and  error  messages  encountered  by  the  Master. 


Figure  5-15.  TMS  Master  Startup  menus 


Operations 

The  TMS  Master  application  currently  supports  four  system  operations:  Exercise,  Survey, 
Playback  and  HLA  simulation.  The  following  sections  provide  overviews  of  the  user  interactions 
in  each  of  these  operations. 

Exercise 

To  initiate  an  exercise,  select  the  ‘Exercise’  item  on  the  ‘Operations’  menu.  This 
selection  invokes  the  ‘Exercise  Setup’  dialog  shown  in  Figure  5-16.  The  setup  dialog  consists  of 
2  tabular  dialogs,  General  and  Entities. 

The  ‘Display’  section  of  the  ‘General’  tab  allows  the  evaluator  to  select  the  application(s) 
that  will  graphically  display  the  exercise’s  position  and  ground  truth  data.  TMS  and  CTMS  are 
the  system  defaults,  but  a  single  application  can  be  chosen.  If  both  applications  are  selected,  the 
‘Start’,  ‘Pause’  and  ‘Stop’  actions  can  be  manipulated  from  either  application  to  control  the 
exercise.  If  TMS  is  not  selected,  TMS  Master  displays  the  selected  exercise  ground  truth  data, 
but  does  not  display  received  position  data  updates. 

To  archive  an  exercise,  check  the  ‘Archive  Exercise’  box.  When  an  exercise  is  archived, 
all  instrumentation  data  received  while  the  exercise  is  running  is  written  to  a  file  in  the 
workstation’s  archive  directory.  The  exercise  filename  is  the  name  of  the  exercise  with  the 
extension  “.ARH.”  The  exercise  name  is  created  using  the  fields  in  the  ‘Exercise  ID’  section  of 
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the  ‘General’  tab.  Using  the  pull-down  lists  under  each  field  header,  the  exercise  name  can  be 
changed. 


Figure  5-16.  Exercise  Setup  dialog 


‘Exercise  Description’  is  a  section  provided  for  entering  a  description  or  comment 
pertaining  to  an  exercise.  If  the  exercise  is  archived,  the  exercise  description  is  written  to  the 
header  of  the  exercise  file. 

In  the  ‘Ground  Truth’  section  of  the  ‘General’  tab,  a  ground  truth  file  must  be  entered. 
The  ground  truth  file  can  be  entered  by  typing  in  a  filename  (including  the  path)  at  the 
‘Filename’  prompt,  or  by  selecting  the  ‘Browse’  button.  The  ‘Browse’  button  launches  a  file 
dialog.  The  ground  truth  file  must  be  in  WILD  format. 

The  ‘Positioning  System’  section  of  the  ‘General’  tab  provides  the  ability  to  configure  the 
potential  positioning  systems  used  in  TMS.  If  ‘Arc  Second’  is  selected,  the  Master  searches  in 
the  workstation’s  archive  directory  for  a  “.REF”  file  with  the  same  name  as  the  ground  truth  file. 
If  the  “.REF”  file  is  not  found,  a  dialog  is  launched  prompting  the  evaluator  to  input  the  Arc 
Second  Universal  Transverse  Mercator  (UTM)  reference  coordinates.  A  “.REF”  file  (with  the 
same  name  as  the  ground  truth  file)  is  created  in  the  workstation’s  archive  directory  containing 
the  reference  coordinates.  These  values  are  used  during  the  exercise  to  translate  the  relative  Arc 
Second  position  data  to  absolute  UTM  positions.  The  default  positioning  system,  ‘Other’,  does 
not  perfonn  any  translation  processing. 
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The  ‘Entities’  tab  shown  in  Figure  5-17  lists  the  available  field  entities  through  which 
exercise  data  can  be  received.  Available  entities  correspond  to  communications  interfaces 
configured  and  set-up  by  TMS  DataComms.  An  exercise  may  have  multiple  entities  selected. 
The  primary  entity  is  the  entity  that  is  used  in  the  coverage  analysis  and  sensor  data  display  of 
the  Master,  and  displayed  on  the  CTMS  workstation.  By  default,  if  there  is  only  1  selected 
entity,  the  primary  entity  is  the  selected  entity.  However,  if  there  is  more  than  1  selected  entity, 
the  evaluator  must  select  the  primary  entity  by  double-clicking  on  the  desired  entity  in  the 
‘Selected  Entities’  list. 


Figure  5-17.  Entities  tab 

After  all  required  parameters  have  been  entered  and  the  ‘OK’  button  has  been  clicked,  the 
Master  changes  its  display,  menus,  toolbar  and  status  bar  as  shown  in  Figure  5-18.  The  first  pane 
on  the  status  bar  displays  the  operation  type.  The  second  pane  displays  the  ground  truth  filename 
and  the  third  pane  is  the  exercise  name.  The  icons  on  the  grid  display  depict  the  objects  in  the 
ground  truth  file. 
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Figure  5-18.  Exercise  operation 

If  CTMS  is  chosen  as  a  display  application,  an  exercise  cannot  begin  until  the  ground 
truth  file  has  been  loaded  in  CTMS  on  the  CTMS  workstation.  On  the  status  bar  of  the  Master 
and  the  CTMSHost  application,  a  flashing  message  alerts  the  evaluator  to  load  the  ground  truth 
file.  Refer  to  Section  5. 3.4. 2.1  for  instructions  on  loading  a  ground  truth  file  in  CTMS.  After  the 
ground  truth  file  is  loaded  in  CTMS,  the  ‘Start’  button  and  the  ‘Start’  item  on  the  ‘Actions’  menu 
are  enabled. 

Before  an  exercise  can  begin,  it  must  also  be  determined  that  the  selected  field  entities  are 
configured  and  ready.  After  the  exercise  is  configured,  if  the  selected  entities  are  not  ready, 
DataComms  sends  a  message  to  the  Master.  When  the  entities  become  ready,  the  ‘Start’  button 
turns  green  and  the  ‘Start’  item  on  the  ‘Actions’  menu  becomes  enabled. 

When  the  ‘Start’  button  turns  green,  the  system  enters  calibration  mode;  real-time 
position  data  is  plotted  for  the  platfonn  and  sensor.  The  calibration  mode  is  used  to  prove  that 
the  positioning  system  is  calibrated  with  respect  to  the  selected  ground  truth.  The  operator  can 
arbitrarily  place  a  position  sensor  at  any  point  relative  to  the  lane  and  the  evaluator’s  display 
should  correctly  indicate  the  relative  position  of  the  position  sensor  and  the  ground  truth  object. 
In  calibration  mode,  no  analysis  is  perfonned. 


Exercise  Functions 

The  following  sections  describe  the  application  controls  and  options  provided  during  an 
exercise  operation. 
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New  Exercise 

To  configure  a  new  exercise,  select  the  ‘New  Exercise’  item  on  the  ‘File’  menu  or  click 
the  toolbar  button  shown  in  Figure  5-19.  This  selection  displays  the  tabular  ‘Exercise  Setup’ 
dialog  shown  in  Figure  5-3. 


□ 

Figure  5-19,  New  Exercise  toolbar  button 


Open  Ground  Truth 

Before  an  exercise  begins,  the  ground  truth  file  can  be  reselected.  To  select  a  new  ground 
truth  file,  select  the  ‘Open  Ground  Truth’  item  on  the  ‘File’  menu  or  click  the  toolbar  button 
shown  in  Figure  5-20.  The  ‘Open  Ground  Truth’  item  launches  a  file  dialog. 


Figure  5-20.  Open  Ground  Truth  toolbar  button 


Application  Display 

Before  an  exercise  begins,  the  selected  application  display(s)  can  be  modified.  The  TMS 
display  and  the  CTMS  display  toolbar  buttons  are  shown  in  Figure  5-21.  When  an  application 
display  is  selected,  its  button  appears  depressed. 
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Figure  5-21.  TMS  and  CTMS  toolbar  buttons 


Archive 

The  ‘Archive’  toolbar  button  shown  in  Figure  5-22  is  a  toggle  button.  The  button  appears 
depressed  when  archiving  is  turned  on.  To  turn  on  archiving,  invoke  the  exercise  name  dialog 
shown  in  Figure  5-23,  by  selecting  the  ‘Archive’  toolbar  button.  Use  the  default  exercise  name 
or  modify  it  using  the  pull-down  lists  under  each  field  header  and  click  ‘OK’. 

$ 

Figure  5-22.  Archive  toolbar  button 
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Figure  5-23.  Exercise  Name  dialog 


View 

The  dimensions  represented  by  the  grid  display  can  be  increased  and  decreased  using  the 
zoom  out  and  zoom  in  operations,  respectively.  Zooming  operations  maintain  the  current  center 
point  of  the  grid  display.  The  zoom  operations  can  be  invoked  in  two  ways.  The  toolbar 
contains  two  toolbar  buttons,  shown  in  Figure  5-24;  one  for  zoom  out  and  one  for  zoom  in. 
Clicking  one  of  these  buttons  changes  the  grid  display  by  one  zoom  increment.  The  ‘View’ 
menu  also  contains  ‘Zoom  In’  and  ‘Zoom  Out’  submenus.  Each  of  these  submenus  contains 
options  for  zooming  by  one  or  multiple  increments. 

£) 

Figure  5-24.  Zoom  Out/Zoom  In  toolbar  buttons 


Coverage  Display 

The  ‘Coverage  Display’  toolbar  button,  shown  in  Figure  5-25,  and  the  ‘Coverage 
Display’  item  on  the  ‘View’  menu  launches  a  modeless  dialog  of  checkboxes  that  toggle  the 
display  of  areas  covered  too  fast,  too  slow  and  areas  covered  okay.  Each  coverage  item  is 
displayed  on  the  grid  display  using  different  colors  and  different  fill  patterns. 


Figure  5-25.  Coverage  Display  toolbar  button 


Display  Options 

The  ‘Display  Options’  toolbar  button,  shown  in  Figure  5-26,  and  the  ‘Display  Options’ 
item  on  the  ‘View’  menu  launches  a  modeless  dialog  of  checkboxes  that  toggle  the  display  of 
grid  lines,  lane  boundaries,  landmarks  and  alarms.  By  default,  each  of  the  objects  is  displayed. 


#1 

Figure  5-26.  Display  Options  toolbar  button 


Handheld  Parameters 


1-54 


To  set  the  parameters  specific  to  a  handheld  system,  select  the  ‘Handheld  Parms’  item  on 
the  ‘View’  menu  or  select  the  toolbar  button  shown  in  Figure  5-27.  The  detector’s  radius, 
minimum  speed  and  maximum  speed  values  can  be  modified.  The  speed  parameters  are  used  in 
depicting  areas  covered  by  the  detector  within  the  Coverage  Display  option. 


Figure  5-27.  Handheld  Parameters  toolbar  button 


Ground  Vehicle  Parameters 

Specific  ground  vehicle  parameters  will  be  added  when  operational  requirements  are 
sufficiently  defined. 


Figure  5-28.  Ground  Vehicle  Parameters  toolbar  button 


Video 

To  begin  processing  video  from  the  primary  entity  during  a  live  exercise,  select  the 
‘Video’  item  on  the  ‘View’  menu  or  select  the  toolbar  button  shown  in  Figure  5-29.  This  action 
creates  a  video  session  and  sends  a  command  to  the  instrumentation  unit  of  the  primary  entity  to 
send  video.  The  video  session  will  be  waiting  until  the  instrumentation  unit  processes  the 
command  and  begins  sending  the  video.  The  video  session  window  is  displayed  in  Figure  5-X.  If 
a  system  alert  indicating  that  the  Video  On  command  was  not  sent  successfully  appears,  reselect 
the  Video  item  on  the  View  menu  or  the  toolbar  button  to  send  the  command  again.  An  active 
video  session  can  exist  across  multiple  consecutive  exercises  with  the  same  primary  entity.  If  the 
video  is  to  be  recorded,  launch  TechSmith  Snaglt.  When  the  exercise  is  started,  the  Snaglt  video 
capture  processing  will  commence.  To  terminate  video,  close  the  video  session  window.  When 
the  exercise  is  stopped  or  the  video  is  terminated  during  the  exercise,  the  Snaglt  video  capture 
will  also  terminate.  Note:  when  prompted  for  a  name  for  the  video  file  produced  by  the  Snaglt 
capture  process,  pressing  the  Ctrl-V  key  combination  will  paste  the  current  exercise  name  into 
the  file  name  text  field  on  the  Save  As  file  dialog. 


□ 


Figure  5-29.  Video  toolbar  button 
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Figure  5-30.  Video  session  window 


Detector  Data 

To  display  the  incoming  metal  detector  data,  select  the  ‘Incoming  Metal  Detector  Data’ 
item  on  the  ‘View’  menu  or  select  the  toolbar  button  shown  in  Figure  5-3 1 .  When  this  selection 
is  made,  the  modeless  dialog  shown  in  Figure  5-32  is  launched  displaying  the  detector  channel 
data  in  strip  charts.  The  maximum  number  and  scaling  of  the  displayed  channels  are  specific  to 
the  type  of  detector  used  in  the  exercise.  To  modify  a  strip  chart’s  parameters,  double-click  on 
the  strip  chart.  The  modifiable  parameters  are  line  width,  line  color,  grid  color,  background 
color,  minimum  Y-axis  and  maximum  Y-axis.  Any  displayed  strip  chart  can  be  disabled 
independently  of  all  others.  Disabling  a  strip  chart  freezes  the  chart  with  the  currently  displayed 
data  and  does  not  update  the  display  with  subsequent  data.  A  strip  chart  is  disabled  by  placing 
the  cursor  on  the  strip  chart  and  typing  ‘d’  and  is  enabled  by  typing  ‘e’.  The  display  of  all 
channels  can  be  paused  by  selecting  the  ‘Pause’  button  on  the  dialog.  A  strip  chart  can  be 
removed  from  the  display  by  de-selecting  the  corresponding  check  box  at  the  top  of  the  dialog. 
The  display  status  and  scale  values  for  each  channel  of  each  supported  type  of  detector  are 
retained  for  subsequent  exercises  and  execution  sessions. 
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Figure  5-31.  Metal  Detector  Data  toolbar  button 


Figure  5-32.  Metal  Detector  Data  Strip  Chart  dialog 


System  Events  Log 

To  display  the  system  events  log,  select  the  ‘TMS  Master  System  Log’  item  on  the 
‘View’  menu  or  select  the  toolbar  button  shown  in  Ligure  5-33.  The  system  events  log  is  a 
modeless  dialog  containing  the  system  event  and  error  messages  encountered  by  the  Master. 


Figure  5-33.  System  Events  Log  toolbar  button 
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Start 


To  start  an  exercise,  select  the  ‘Start’  item  on  the  ‘Actions’  menu  or  select  the  toolbar 
button  shown  in  Figure  5-34.  When  an  exercise  is  started,  the  ‘Start’  action  is  disabled  and  the 
‘Pause’  and  ‘Stop’  actions  are  enabled. 


Figure  5-34.  Start  toolbar  button  (green) 


Pause 

After  an  exercise  is  started,  the  ‘Pause’  toolbar  button,  shown  in  Figure  5-35,  and  the 
‘Pause’  item  on  the  ‘Actions’  menu  are  enabled.  When  an  exercise  is  paused,  the  ‘Stop’  action  is 
disabled.  In  paused  mode,  the  position  of  the  platfonn  and  sensor  continue  to  be  displayed,  but 
in  a  different  color.  No  evaluation  processing  occurs  during  paused  mode.  To  resume  a  paused 
exercise,  reselect  ‘Pause’. 


Figure  5-35.  Pause  toolbar  button  (yellow) 


Stop 

The  ‘Stop’  toolbar  button  shown  in  Figure  5-36  and  the  ‘Stop’  item  on  the  ‘Actions’ 
menu  terminates  an  exercise.  A  stopped  exercise  cannot  be  resumed;  hence,  a  confirmation  box 
is  displayed  for  the  evaluator  to  confirm/cancel  the  ‘Stop’  action.  Included  in  the  confirmation 
box  are  the  following  options:  power  down  instrumentation,  delete  exercise  file,  delete  exercise. 
After  the  exercise  is  stopped,  the  selected  options  are  performed.  The  confirmation  box  is  not 
displayed  when  stopping  a  playback  exercise. 


Figure  5-36.  Stop  toolbar  button  (red) 


Restart 

The  ‘Restart’  toolbar  button  shown  in  Figure  5-37  and  the  ‘Restart’  item  on  the  ‘Actions’ 
menu  are  used  to  configure  an  exercise  using  the  setup  parameters  of  the  previous  exercise.  If 
the  previous  exercise  was  archived,  the  exercise  name  dialog  shown  in  Figure  5-23  is  displayed 
when  ‘Restart’  is  selected. 


M 

Figure  5-37.  Restart  toolbar  button 


Track  Primary  Entity 

To  maintain  the  display  of  the  primary  entity  within  the  current  grid  display,  select  the 
‘Track  Primary  Entity’  item  on  the  ‘View’  menu. 
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Entity  List 

To  display  the  available  and  selected  entities  in  an  exercise,  select  the  ‘View  Entity  List’ 
item  on  the  ‘Entities’  menu.  This  selection  launches  a  dialog  similar  to  Figure  5-17.  Before  an 
exercise  begins,  the  selected  and  primary  entity  can  be  modified.  After  an  exercise  begins,  the 
selected  entities  can  be  reviewed,  but  not  modified. 

Set  CTMS  Data  Rate 

When  CTMS  display  mode  is  selected,  the  position  data  passed  to  CTMS  Host  for 
processing  by  CTMS  can  be  decimated  by  a  factor  set  using  the  Set  CTMS  Data  Rate  item  of  the 
File  menu.  Selecting  this  item  displays  the  CTMS  Position  Data  Rate  Reduction  dialog  shown  in 
Figure  5-38.  This  dialog  allows  the  user  to  specify  data  rate  reduction  (decimation)  factors  that 
are  applied  separately  to  the  platfonn  (feet)  and  detector  head  position  data.  The  rate  at  which  the 
corresponding  data  is  sent  to  CTMS  Host  is  the  rate  the  data  is  received  from  the  FIU  reduced  by 
the  specified  factors.  For  example,  if  the  position  data  is  received  at  a  rate  of  10  Hz,  a  factor  of 
10  will  reduce  the  data  transfer  rate  to  1  Hz.  Setting  the  rate  factor  control  to  the  “No  Data” 
position  will  stop  the  transfer  of  that  data.  Selecting  the  “Default”  button  will  restore  both  of  the 
factors  to  the  original  recommended  values.  The  factor  values  specified  using  this  feature  are 
used  for  subsequent  exercises  and  execution  sessions. 


Figure  5-38.  CTMS  Position  Data  Rate  Reduction  dialog 


Detector  Head  Velocity 

Once  position  data  from  the  exercise  primary  entity  is  available,  a  display  in  strip  chart 
format  of  the  detector  head  velocity  can  be  accessed  using  the  Primary  Detector  Velocity  item  of 
the  View  menu.  This  display,  shown  in  Figure  5-39,  plots  the  calculated  velocity  of  the  detector 
head  versus  time.  The  strip  chart  background  is  divided  into  three  bands  based  on  the  detector 
head  minimum  and  maximum  speed  values  referenced  in  Section  5. 3.2. 2. 1 . 1 .8.  These  bands 
correspond  to  the  assessment  of  head  velocity  depicted  with  the  Coverage  Display  option:  too 
slow,  OK,  and  too  fast.  The  currently  entered  values  for  minimum  and  maximum  speed  are 
indicated  at  the  top  of  the  display.  The  vertical  axis  scaling  for  this  display  can  be  adjusted  using 
the  technique  described  in  Section  5. 3.2. 2. 1.1.1 1. 
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Figure  5-39.  Detector  Head  Velocity  dialog 


Playback 

To  initiate  Playback,  select  the  ‘Playback’  item  on  the  ‘Operations’  menu.  This  selection 
displays  the  file  dialog  shown  in  Figure  5-40.  By  default,  the  dialog  lists  exercise  files  (with  a 
“.ARFI”  extension)  in  the  workstation’s  archive  directory.  To  view  the  header  information  of  an 
exercise  file,  highlight  the  filename  by  placing  the  cursor  over  the  name  and  left  clicking.  The 
header  information  contains  the  exercise  name,  exercise  description,  ground  truth  name,  location 
of  ground  truth  file,  number  of  entities,  duration  of  exercise  and  for  each  entity:  ID,  type  and 
number  of  components.  If  the  highlighted  file  is  not  in  the  correct  format,  an  error  message  will 
be  displayed.  To  select  a  file  for  playback,  select  ‘Open’. 


Figure  5-40.  Exercise  File  Selection  dialog 
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After  the  file  is  selected,  the  evaluator  is  prompted  to  select  which  application(s),  TMS 
and/or  CTMS,  will  graphically  display  the  exercise’s  position  and  ground  truth  data.  Both  TMS 
and  CTMS  are  the  defaults,  but  a  single  application  can  be  chosen.  If  both  applications  are 
selected,  the  ‘Start’,  ‘Pause’,  ‘Stop’,  and  fast-forward  actions  can  be  manipulated  from  either 
application  to  control  the  exercise.  If  TMS  is  not  selected,  TMS  displays  the  exercise’s  ground 
truth  data,  but  does  not  display  position  updates. 

As  shown  in  Figure  5-41,  the  Master  changes  its  display,  menus,  toolbar  and  status  bar 
after  the  file  is  opened.  The  status  bar  panes  definitions  in  Playback  are  identical  to  their 
Exercise  definitions.  The  first  pane  on  the  status  bar  displays  the  operation.  The  second  pane 
displays  the  ground  truth  filename  and  the  third  pane  is  the  exercise  name.  The  icons  on  the  grid 
display  depict  the  objects  in  the  ground  truth  file. 


Figure  5-41.  Playback  operation 

If  CTMS  is  chosen  as  a  display  application,  a  playback  exercise  cannot  begin  until  the 
ground  truth  file  has  been  loaded  in  CTMS  on  the  CTMS  workstation.  On  the  status  bars  of  the 
Master  and  the  CTMSFIost  applications,  a  flashing  message  alerts  the  evaluator  to  load  the 
ground  truth  file.  Refer  to  Section  5. 3. 4.2.1  for  instructions  on  loading  a  ground  truth  file  in 
CTMS.  After  the  ground  truth  file  is  loaded  in  CTMS,  the  ‘Start’  button  and  the  ‘Start’  item  on 
the  ‘Actions’  menu  are  enabled. 

Playback  Functions 
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The  following  sections  describe  the  application  controls  and  options  provided  during  a 
playback  operation. 

Open  Exercise  File 

To  configure  another  playback  exercise,  select  the  ‘Open  Exercise  File’  item  on  the  ‘File’ 
menu  or  click  the  toolbar  button  shown  in  Figure  5-20.  This  will  display  the  file  dialog  shown  in 
Figure  5-40.  Select  a  file  and  click  ‘Open’. 

Application  Display 

Refer  to  Section  5. 3. 2.2. 1.1. 3. 


View 

Refer  to  Section  5. 3. 2.2. 1.1. 5. 

Coverage  Display 

Refer  to  Section  5. 3. 2.2. 1.1. 6. 

Display  Options 

Refer  to  Section  5. 3. 2.2. 1.1. 7. 

Handheld  Parameters 

Refer  to  Section  5. 3. 2. 2. 1.1. 8. 

Ground  Vehicle  Parameters 

Refer  to  Section  5. 3. 2.2. 1.1. 9. 


Video 

The  ‘Video’  item  of  the  ‘View’  menu  allows  the  user  to  select  a  video  file  (.AVI  format) 
recorded  during  a  live  exercise  to  play  during  the  playback  of  the  archived  exercise  data.  The 
recorded  video  display  is  controlled  along  with  the  position  and  instrumentation  data  by  the 
playback  controls.  The  video  display  window  can  be  closed  at  any  time,  but  a  video  file  can  be 
selected  only  prior  to  starting  the  exercise  data  playback. 

Detector  Data 

Refer  to  Section  5.3.2.2.1.1.11. 

System  Events  Fog 

Refer  to  Section  5.3.2.2.1.1.12. 


Start 

Refer  to  Section  5.3.2.2.1.1.13. 


Pause 

Refer  to  Section  5.3.2.2.1.1.14. 
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Stop 


Refer  to  Section  5.3.2.2.1.1.15. 


Restart 

The  ‘Restart’  toolbar  button  shown  in  Figure  5-36  and  the  ‘Restart’  item  on  the  ‘Actions’ 
menu  are  used  to  configure  a  playback  exercise  using  the  same  exercise  file  as  the  previous 
playback  exercise. 


Fast  Forward 

After  a  playback  exercise  is  started,  the  display  rate  can  be  changed  using  the  fast 
forward  control,  shown  in  Figure  5-42.  Using  the  combo  box,  highlight  the  desired  value  then 
left  click  the  double  arrow  toolbar  button.  This  button  will  remain  in  the  “down”  condition  while 
in  fast  forward  mode.  The  display  rate  of  the  exercise  file  is  divided  by  the  selected  value.  While 
in  fast  forward  mode,  the  rate  can  be  changed  by  selecting  another  value  in  the  combo  box.  To 
return  to  the  original  display  rate  of  the  exercise  file,  click  the  double  arrow  toolbar  button  again. 


2. Ox 


4. Ox 
6. Ox 
8. Ox 


Figure  5-42.  Fast  Forward  control 


Track  Primary  Entity 

Refer  to  Section  5.3.2.2.1.1.17. 

Entity  List 

To  view  the  entity(s)  in  a  playback  exercise,  select  the  ‘View  Entity  List’  item  on  the 
‘Entities’  menu. 

Set  CTMS  Data  Rate 

Refer  to  Section  5.3.2.2.1.1.19. 


HLA 

To  initiate  an  HLA  exercise  in  TMS,  select  the  ‘HLA’  item  on  the  ‘Operations’  menu. 
This  selection  invokes  the  ‘HLA  Exercise  Setup’  dialog  shown  in  Figure  5-43.  The  setup  dialog 
consists  of  2  tabular  dialogs,  General  and  Reference  Point. 
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Figure  5-43.  HLA  Exercise  Setup  dialog 

On  the  ‘General’  tab,  enter  a  ‘Federation  Name’  or  use  the  default,  TMSFED.  The 
federation  name  identifies  the  FED  file.  The  FED  file  is  an  ASCII  file  containing  the  Federation 
Object  Model  (FOM).  The  FOM  used  by  TMS  is  RPRFOM  1.0.  As  a  part  of  the  TMS 
installation,  the  FED  file,  TMSFED. fed,  is  copied  to  the  DataComms  executable  directory.  If  the 
default  federation  name  is  not  used,  a  FED  file  must  be  created  and  placed  in  the  DataComms 
executable  directory.  The  FED  file  naming  convention  is  FederationName.fed. 

To  archive  an  HLA  exercise,  check  the  box  ‘Archive  Exercise’.  Currently,  only  the 
federation  name  and  the  ground  truth  filename  are  written  to  the  exercise  file  in  the  workstation’s 
archive  directory.  The  exercise  filename  is  the  name  of  the  exercise  with  the  extension  .HLA. 
An  HLA  exercise  name  is  the  federation  name  plus  a  date  and  timestamp. 

On  the  ‘General’  tab,  an  exercise  entity  type  must  be  selected.  In  an  HLA  exercise,  TMS 
can  interact  with  the  following  entity  types:  troops,  wheeled  vehicles  or  tracked  vehicles.  A 
troop  may  consist  of  many  soldiers  or  a  single  soldier. 

An  HLA  exercise  requires  an  initial  position,  a  reference  point,  with  which  to  initialize 
the  orientation  of  the  grid  display  and  the  ground  truth  file.  The  reference  point  is  derived  from 
the  object  data  in  the  ground  truth  file.  On  the  ‘Reference  Point’  tab  shown  in  Figure  5-44,  enter 
the  ground  truth  filename  in  the  ‘Ground  Truth  Name’  field  or  select  ‘Browse’  to  invoke  a  file 
dialog.  The  ground  truth  data  must  be  in  WILD  format.  The  ground  truth  data  only  specifies 
UTM  easting  and  northing  components,  hence,  the  UTM  zone  number  and  hemisphere  must  be 
provided  on  the  ‘Reference  Point’  tab. 
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HLA  Exercise  Setup 


General  Reference  Point 


Ground  Truth  Filename 


Hemisphere 


2<J 


Browse 


UTM  Zone  Number: 

f*  North  C  South 


OK 


Cancel 


Apply 


Figure  5-44.  Reference  Point  tab 

As  shown  in  Figure  5-45,  the  Master  changes  its  display,  menus,  toolbar  and  status  bar 
after  all  the  required  parameters  have  been  entered  and  the  ‘OK’  button  is  selected.  The  first 
pane  on  the  status  bar  displays  the  operation.  The  second  pane  displays  the  ground  truth 
filename  and  the  third  pane  is  the  exercise  name.  The  icons  on  the  grid  display  depict  the  objects 
in  the  ground  truth  file. 
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Figure  5-45.  HLA  operation 

If  the  DataComms/DSP  computer  has  2  network  interfaces,  the  RTI  must  be  configured 
before  executing  (creating/joining)  an  HLA  exercise.  TMS  uses  RTI1.3NGv4.  To  configure  the 
RTI,  edit  the  ‘RTI. rid’  file.  As  a  part  of  the  TMS  installation,  there  should  be  an  ‘RTI. rid’  file  in 
the  DataComms  executable  directory.  In  the  RID  file,  modify  the  line: 
;;(RTIExecutiveEndpoint  hostname:port).  The  semicolons  serve  as  comment  tokens  and  must  be 
removed.  The  hostname  is  the  IP  address  or  hostname  of  the  federate  that  will  execute  the 
‘rtiexec’  and  port  is  a  valid  port  number.  The  federate  executing  the  ‘rtiexec’  should  invoke  the 
‘rtiexec’  as  follows:  rtiexec  -endpoint  hostname:port.  The  values  of  hostname  and  port  are 
identical  to  the  values  entered  in  the  RID  file. 

HLA  Functions 

The  following  sections  describe  the  application  controls  and  options  provided  during  an 
HLA  exercise  operation. 


New  Exercise 

To  configure  a  new  HLA  exercise,  select  the  ‘New  Exercise’  item  on  the  ‘File’  menu  or 
click  the  toolbar  button  shown  in  Figure  5-19.  This  selection  will  display  the  HLA  Exercise 
Setup  dialog  shown  in  Figure  5-43. 

Open  Ground  Truth 
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Before  an  HLA  exercise  begins,  the  ground  truth  file  can  be  reselected.  To  select  a  new 
ground  truth  file,  select  the  ‘Open  Ground  Truth’  item  on  the  ‘File’  menu  or  click  the  toolbar 
button  shown  in  Figure  5-20.  This  selection  will  launch  the  dialog  shown  in  Figure  5-44. 
Provide  a  ground  truth  name,  zone  number  and  hemisphere  and  click  ‘OK’. 

Archive 

The  ‘Archive’  toolbar  button  shown  in  Figure  5-22  is  a  toggle  button.  If  archiving  is 
selected,  the  button  appears  depressed.  To  turn  on  archiving,  click  the  ‘Archive’  toolbar  button. 

View 

Refer  to  Section  5. 3. 2.2. 1.1. 5. 

Create  and  Join 

To  start  an  FILA  exercise,  select  the  ‘Create  and  Join’  item  on  the  ‘Actions’  menu  or 
select  the  toolbar  button  shown  in  Figure  5-46.  When  “Create  and  Join’  is  selected,  the  evaluator 
is  asked  to  verify  that  the  ‘rtiexec’  is  running.  If  the  ‘rtiexec’  is  not  running,  DataComms  will 
crash  when  TMS  tries  to  create  the  federation.  When  TMS  joins  a  federation,  the  ground  truth 
data  is  published  to  the  other  federates. 


r 

Figure  5-46.  Create  and  Join  toolbar  button 


Resign  and  Destroy 

The  ‘Resign  and  Destroy’  toolbar  button  shown  in  Figure  5-47  and  the  ‘Resign  and 
Destroy’  item  on  the  ‘Actions’  menu  is  enabled  after  an  HLA  exercise  begins.  To  end  the 
participation  of  TMS  in  an  HLA  exercise,  select  the  ‘Resign  and  Destroy’  item. 


s 


Figure  5-47.  Resign  and  Destroy  toolbar  button 


System  Events  Log 

Refer  to  Section  5.3.2.2.1.1.12. 
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Track  Primary  Entity 

Refer  to  Section  5.3.2.2.1.1.17. 

Entity  List 

To  display  the  selected  entity  in  an  HLA  exercise,  select  the  ‘View  Entity  List’  item  on 
the  ‘Entities’  menu.  Before  an  HLA  exercise  begins,  the  selected  entity  can  be  modified.  After 
an  HLA  exercise  begins,  the  entities  can  be  viewed,  but  not  modified. 

Survey 

The  Survey  operation,  in  conjunction  with  commercially  available  field  survey 
equipment,  provides  an  automated  method  of  developing  and  graphically  modifying  ground  truth 
files.  Ground  truth  files  may  be  created  using  survey  point  data  transmitted  wirelessly  from  one 
or  more  surveyors  in  the  field  or  from  data  entered  manually.  To  initiate  a  survey  session,  select 
the  ‘Survey’  item  on  the  ‘Operations’  menu.  This  presents  a  submenu  consisting  of  2  items: 
‘New  Survey...’  and  ‘Open  Survey  Data’.  Select  the  ‘New  Survey  Data...’  item  to  start  a  new 
survey  or  to  resume  using  the  data  from  the  immediately  previous  survey  session.  Selecting  the 
‘Open  Survey  Data’  item  presents  another  submenu  consisting  of  2  items:  ‘Ground  Truth  File. . .  ’ 
and  ‘Saved  Survey  Data...’.  Select  the  ‘Ground  Truth  File...’  item  to  open  an  existing  ground 
truth  file  in  the  WILD  format  generated  from  a  previous  survey  session  or  by  other  methods 
implementing  the  WILD  format.  Select  the  ‘Saved  Survey  Data...’  item  to  open  the  data 
generated  from  any  previous  survey  session.  Initiating  a  survey  session  will  display  the  Survey 
operation  main  window  shown  in  Figure  5-48.  This  window  includes  a  grid  display  for 
graphically  depicting  the  surveyed  area  and  objects;  a  variable  content  display  that  optionally 
displays  lists  of  object  data,  position  message  data,  and  reference  point  data;  and  a  running  list  of 
system  events. 

The  Survey  operation  requires  an  initial  position  with  which  to  initialize  the  orientation 
of  the  grid  display  and  other  system  components.  This  position  data  is  entered  at  the  start  of  a 
new  survey  session  in  the  Survey  Data  dialog  shown  in  Figure  5-49.  The  data  required  consists 
of  a  UTM  grid  number,  a  grid  easting,  a  grid  northing,  an  altitude,  and  the  selection  of  the 
northern  or  southern  hemisphere.  After  this  data  is  entered  into  the  system,  the  equivalent 
latitude  and  longitude  (in  degrees)  of  the  reference  point  will  also  be  provided  in  the  survey  data 
display.  The  reference  point  position  data  can  be  modified  at  any  time  during  a  survey  session. 
An  initial  object  identification  number  as  well  as  initial  lane  identification  data  consisting  of  a 
lane  number  and  a  lane  description  is  also  required.  Additional  lane  identification  data  can  also 
be  added  at  any  time  during  a  survey  session. 
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Figure  5-48.  Survey  operation  initial  main  window 


The  Survey  Data  dialog  also  contains  a  “Log  survey  data”  check  box.  Selecting  this  check  box 
will  log  all  of  the  raw  data  received  from  field  surveyors  in  a  file  for  later  processing  and  review. 
This  log  file  is  not  required  for  any  Survey  application  function. 
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Enter  Survey  Data 


Figure  5-49.  Enter  Survey  Data  dialog 

The  Survey  operation  receives  and  processes  position  data  generated  at  and  transferred 
from  field  surveyors.  This  data  is  transferred  in  the  form  of  messages  sent  over  wireless 
communications  channels.  A  corresponding  communications  interface  of  the  appropriate  type 
must  be  configured  within  TMS  DataComms  prior  to  initiating  survey  operations  with  a  field 
surveyor.  A  descriptive  label  entered  as  part  of  the  interface  configuration  information  is  used 
within  the  Survey  operation  to  identify  the  field  surveyor.  In  order  to  receive  position  messages 
from  field  surveyors,  one  or  more  configured  survey  communications  interfaces  must  be  selected 
by  the  user.  Surveyors  are  selected  using  the  ‘Select  Surveyor’  item  of  the  ‘Conuno’  menu. 
Selecting  this  item  will  display  the  Select  Surveyors  dialog  shown  in  Figure  5-50.  This  dialog 
presents  a  list  of  available  survey  communications  interfaces  (“surveyors”)  as  established  by 
TMS  DataComms  as  well  as  a  list  of  surveyors  already  allocated  to  the  current  survey  session. 
Surveyors  are  moved  on  and  off  each  list  using  the  ‘Add’  and  ‘Remove’  buttons,  as  appropriate. 
Surveyors  may  be  selected  and  de-selected  for  a  survey  session  at  any  time,  provided  that  the 
corresponding  communications  interface  is  available  through  TMS  DataComms. 
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Select  Surveyors 


Available  Surveyors 

1 5-team  2 

6-dummy  surv 


ADD 


REMOVE 


Selected  Surveyors 


3-Survey  Team  1 


OK 


Cancel 


Figure  5-50.  Select  Surveyors  dialog 


When  multiple  surveyors  are  selected  for  a  survey  session,  one  must  be  designated  as 
“current.”  The  current  surveyor  is  that  from  which  the  user  will  request  and  process  position  data 
messages.  The  surveyor  designated  as  current  is  indicated  by  a  unique  icon  and  is  also  indicated 
in  a  pane  of  the  status  bar  at  the  bottom  of  the  main  window.  Figure  5-51  shows  the  icons  for  the 
current  and  active  surveyors  The  current  surveyor  may  be  set  from  among  all  selected  surveyors 
at  any  time  using  the  ‘Current  Surveyor’  item  of  the  ‘Commo’  menu,  which  presents  the  Select  a 
Surveyor  dialog  shown  in  Figure  5-52,  or  by  double-clicking  on  the  icon  of  the  desired  surveyor. 


Figure  5-51.  Current  and  Active  Surveyor  icons 
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Select  A  Surveyor 


Figure  5-52.  Select  a  Surveyor  dialog 

The  type  of  objects  that  may  be  created  within  the  Survey  operation,  and  their 
corresponding  icons,  consist  of  the  following: 


Mines 


/V\ 


Lane  Boundaries 

Landmarks 

Other 

Unknowns 

Monuments 


Alarms  indicating  mines 


Alarms  indicating  clutter 


The  mine  types  and  landmark  identifiers  available  for  assignment  are  extracted  from  the 
TMS/CTMS  database  to  ensure  consistency  with  other  TMS  functions.  (Note:  objects  identified 
as  monuments  and  alarms  are  not  included  in  ground  truth  files.  Only  objects  identified  as  alarms 
are  included  in  alann  files.) 
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Survey  Views 

The  application  main  window  for  the  Survey  operation  includes  multiple  views.  The  grid 
view  and  the  system  event  list  are  always  visible.  However,  the  survey  data  view,  the  object  list 
and  the  position  message  list  share  a  window  and  are  swapped  based  on  actions  of  the  user. 
Figure  5-53  shows  an  example  of  the  main  window  during  a  survey  operation.  The  window  in 
this  example  is  displaying  the  grid  view,  the  position  message  list,  and  the  system  event  list.  In 
addition,  while  in  the  Survey  operation,  the  panes  of  the  main  window’s  status  bar  provide  the 
following  information:  the  current  operation  is  SURVEY;  the  latest  GPS  mode  and  HDOP 
values  received  from  the  current  surveyor  (“MODE/HDOP”  indicates  no  communications;)  the 
ID  of  the  current  surveyor  and  the  number  of  selected  surveyors;  the  TMS  configuration 
(DISTRIBUTED,  STANDALONE,  etc.;)  and  the  usage  percentage  of  the  archive  volume. 


Figure  5-53.  Example  survey  main  window 


Grid  View 

The  grid  view  provides  a  graphical  representation  of  the  relative  positions  of  the  surveyed 
area  and  identified  objects.  The  area  of  the  survey  is  depicted  as  a  section  of  a  UTM  grid,  with 
easting  values  corresponding  to  the  vertical  lines  and  northing  values  corresponding  to  the 
horizontal  lines.  The  grid  is  oriented  with  north  at  the  top  and  is  initially  centered  at  the  reference 
point.  The  current  dimensions  of  the  area  depicted  in  the  grid  view  are  displayed  in  the 
information  pane  (left-most  section)  of  the  status  bar  at  the  bottom  of  the  application  window. 
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When  an  object  is  entered  into  the  system,  an  icon  is  displayed  in  the  grid  view  at  a  location 
corresponding  to  the  object’s  position.  An  icon  is  also  displayed  to  represent  the  current  position 
of  active  surveyors  as  detennined  from  messages  received  from  the  field  equipment.  Clicking  the 
left  mouse  button  once  with  the  pointer  over  an  empty  section  of  the  view  will  enter  the 
corresponding  position  in  the  system  view.  A  number  of  operations  affect  the  area  depicted  by 
the  grid  view.  The  following  sections  describe  these  operations. 

Grid  View  Zooming 

The  area  depicted  by  the  grid  view  can  be  increased  and  decreased  using  the  zoom  out 
and  zoom  in  operations,  respectively.  Zooming  operations  maintain  the  current  center  point  of 
the  view.  The  zoom  operations  can  be  invoked  in  two  ways.  The  toolbar  contains  two  zoom 
buttons,  shown  in  Figure  5-54;  one  for  zoom  in  and  one  for  zoom  out.  Clicking  one  of  these 
buttons  changes  the  area  depicted  by  the  grid  view  by  one  zoom  increment.  The  ‘View’  menu 
also  contains  ‘Zoom  In’  and  ‘Zoom  Out’  submenus.  Each  of  these  submenus  contains  options  for 
zooming  by  one  or  multiple  increments.  After  zooming,  the  current  dimensions  of  the  area 
depicted  in  the  grid  view  are  updated  in  the  information  pane  of  the  status  bar  at  the  bottom  of 
the  application  window. 

Figure  5-54.  Zoom  In/Zoom  Out  toolbar  buttons 


Grid  View  Panning 

Holding  down  the  right  mouse  button  and  moving  the  mouse  pans  over  the  area  depicted 
by  the  grid  view.  When  in  panning  mode,  the  mouse  pointer  changes  to  a  panning  pointer.  When 
the  right  mouse  button  is  released,  panning  mode  is  terminated  and  the  pointer  changes  back  to 
the  standard  pointer.  Panning  will  change  the  center  point  of  the  view. 

Grid  View  Centering 

The  location  of  the  center  of  the  grid  view  can  be  set  corresponding  to  several  positions. 
The  ‘View’  menu  contains  a  ‘Center  Grid  On’  submenu.  The  items  of  this  submenu  will  center 
the  grid  view  on  one  of  the  following:  the  reference  point;  the  current  position  of  the  surveyor; 
the  currently  selected  object  icon;  or  the  icon  of  an  object  selected  from  a  list  of  all  objects 
entered  in  the  system.  In  addition,  the  ‘Center  Grid  On’  submenu  contains  an  option  to  track  the 
current  surveyor’s  position.  After  selecting  this  option,  the  current  surveyor’s  position  is 
maintained  at  the  grid  center  as  the  current  surveyor  moves.  Centering  the  grid  on  either  the 
current  surveyor’s  position  or  tracking  the  surveyor’s  position  can  also  be  invoked  using  the 
‘Track  Surveyor’  toolbar  button,  shown  in  Figure  5-55.  This  toolbar  button  is  “sticky”:  clicking 
the  button  once  puts  the  grid  view  in  track  surveyor  mode;  clicking  it  again  takes  the  view  out  of 
track  surveyor  mode. 


Figure  5-55.  Track  Surveyor  toolbar  button 
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System  Event  List 

The  system  event  list  provides  a  running  log  of  survey  operations.  Survey  operations 
consist  of  receiving  position  messages,  creating  new  objects,  deleting  objects,  modifying  objects, 
etc.  For  an  example  of  the  system  view,  see  Figure  5-53. 

Survey  Data  View 

The  survey  data  view  contains  the  reference  point  position  data  and  the  lane  identification 
data.  To  activate  the  survey  data  view,  select  the  ‘Survey  Data’  item  of  the  ‘View’  menu  or  click 
the  toolbar  button  shown  in  Figure  5-56.  For  an  example  of  the  survey  data  view,  see  Figure  5- 
53.  In  this  view,  the  reference  point  position  data  includes  a  UTM  grid  letter,  which  is  a 
component  of  a  fully  qualified  UTM  position,  as  opposed  to  simply  a  hemisphere  indicator.  The 
‘Modify’  button  at  the  bottom  of  the  view  is  used  to  change  any  of  the  survey  data.  Clicking  this 
button  will  open  the  Survey  Data  dialog,  shown  in  Figure  5-57,  in  which  any  of  the  data  values 
can  be  edited.  The  lane  number  list  contains  lane  numbers  that  have  been  entered  previously.  To 
specify  a  new  lane  number,  type  the  number  into  the  list  box.  The  lane  description  corresponds  to 
the  displayed  lane  number.  This  dialog  also  contains  a  ‘Message’  button.  When  position  data 
messages  are  being  received  from  the  current  surveyor,  clicking  this  button  will  set  the  reference 
point  to  the  position  reported  by  the  surveyor.  This  button  is  disabled  when  position  data 
messages  are  not  being  received  from  the  current  surveyor. 


Figure  5-56.  Survey  Data  View  toolbar  button 
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Survey  Data 


Latitude  (deg): 
Longitude  (deg): 
UTM  Zone  Number: 
UTM  Zone  Letter: 
Northing  (m): 

Easting  (m): 

Altitude  (m): 

Lane  Number: 

Lane  Description: 


16 

S 

3842010.00 

531010.00 

100.00 

- 3 

GRAVEL 
Message 


OK 


Cancel 


Figure  5-57.  Survey  Data  modification  dialog 


Object  List 

The  object  list  is  a  cumulative  list  of  objects  in  the  current  survey  session.  The  object  list 
displays  the  object’s  number,  ID  and  description.  Double-clicking  on  a  list  item  opens  a  dialog 
presenting  all  of  the  information  on  the  corresponding  object.  The  object  list  can  be  sorted  by 
object  number  or  by  ID.  To  sort  the  object  list,  click  the  object  number  or  ID  column  heading. 
The  column  heading  that  is  selected  is  used  as  the  primary  sort  key.  To  activate  the  object  list, 
select  the  ‘Object  List’  item  of  the  ‘View’  menu  or  click  the  toolbar  button  shown  in  Figure  5-58. 
For  an  example  of  the  object  list,  see  Figure  5-59. 


Figure  5-58.  Object  List  toolbar  button 
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Figure  5-59.  Object  List  example 


Position  Message  List 

The  position  message  list  is  a  cumulative  list  of  position  messages  received  during  the 
survey  session.  The  list  view  includes  a  pop-up  list  of  surveyors  that  allows  the  user  to  limit  the 
messages  displayed  to  only  those  from  the  selected  surveyor.  The  position  message  list  displays 
the  position  message’s  number,  status,  and  northing  and  easting  values.  The  possible  message 
status  values  are:  Received,  Read,  Assigned  (the  first  point  assigned  to  an  object),  Assigned+ 
(one  of  several  points  assigned  to  an  object).  To  activate  the  position  message  list,  select  the 
‘Received  Message  List’  item  of  the  ‘View’  menu  or  click  the  toolbar  button  shown  in  Figure  5- 
60.  For  an  example  of  the  position  message  list,  see  Figure  5-53. 


Figure  5-60.  Position  Message  List  toolbar  button 


Survey  Functions 

The  following  sections  describe  the  application  controls  and  options  provided  during  a 
survey  operation. 
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Processing  Position  Messages 

Position  messages  are  obtained  at  the  request  of  the  user  from  the  active  surveyor 
designated  as  current.  When  the  requested  position  message  is  received,  the  user  may  perform  a 
sequence  of  operations  on  the  data  contained  in  the  message. 

Requesting  Position  Messages 

To  request  a  position  message,  select  the  ‘Get  New  Message’  item  of  the  ‘Edit’  menu  or 
click  the  toolbar  button  shown  in  Figure  5-61.  If  a  position  message  is  available,  the  position 
message  is  retrieved  and  added  to  the  position  message  list.  The  messages  in  the  list  will  be 
automatically  limited  to  those  from  the  current  surveyor.  Also,  a  time-stamped  message  of  this 
operation  is  written  to  the  system  event  list.  If  a  position  message  is  not  available,  “No  message 
available”  is  written  to  the  system  event  list.  If  no  surveyors  are  selected  or  no  communications 
are  available  with  the  selected  surveyors,  the  ‘Get  New  Message’  menu  item  and  toolbar  button 
are  disabled. 


Figure  5-61.  Get  New  Message  toolbar  button 


Viewing  a  Position  Message 

To  view  a  position  message,  activate  the  position  message  list,  following  the  instructions 
in  Section  5. 3.2. 2.4. 1.5,  then  left  click  on  the  corresponding  item  in  the  message  list.  The 
displayed  position  data  may  be  used  in  creating  a  new  object,  added  to  an  existing  object,  or  used 
to  modify  an  existing  object.  Figure  5-62  shows  the  Position  Message  dialog. 
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Figure  5-62.  Position  Message  dialog 


Deleting  a  Position  Message 

To  delete  a  position  message,  view  the  position  message,  following  the  instructions  in 
Section  5. 3.2. 2.4. 2. 1.2,  then  click  the  ‘Delete  Message’  button. 

Creating  an  Object 

An  object  may  be  created  using  any  of  the  methods  described  in  the  following  sections. 
From  a  Position  Message 

To  create  an  object  from  a  position  message,  view  the  position  message,  following  the 
instructions  in  Section  5. 3.2. 2.4.2. 1.2,  and  then  click  the  ‘Create  New  Object’  button.  In  the 
resulting  Create  Object  with  Position  Message  dialog,  shown  in  Figure  5-63,  select  the  desired 
object  type.  Once  the  object  type  is  selected,  only  the  associated  object  type  information 
controls  for  that  object  type  are  enabled  and  the  position  message  data  is  displayed  in  the 
Position  Data  section.  Using  this  method  of  creating  an  object,  the  Position  Data  cannot  be 
modified  at  that  time.  All  enabled  text  fields  except  ‘Comments’  require  data.  The  user  may  opt 
to  use  several  position  messages  to  generate  a  weighted  average  position  for  the  object.  To  do  so, 
click  the  ‘Avg’  button  next  to  the  corresponding  position  point.  This  will  open  the  Object 
Position  Data  for  Point  dialog  shown  in  Figure  5-64.  The  first  item  in  the  list  in  this  dialog  will 
be  the  original  position  message  associated  with  the  object  position  point.  To  collect  additional 
messages  for  this  point,  click  the  ‘Record’  button.  This  will  add  received  position  messages  to 
the  list.  Up  to  20  messages  may  be  recorded  for  a  single  point.  To  stop  collecting  position 
messages  click  the  ‘Stop’  button  (which  replaces  the  ‘Record’  button  while  recording.)  To 
calculate  a  weighted  average  position  from  all  of  the  collected  messages,  click  the  ‘Apply’ 
button.  To  remove  a  message,  select  the  message  in  the  list  and  click  the  ‘Clear’  button.  When  a 
weighted  average  position  is  calculated  for  a  point,  the  ‘Avg’  checkbox  associated  with  the  point 
data  in  the  Object  dialog  is  checked.  To  create  the  object  using  the  data  displayed  in  the  Object 
dialog,  click  ‘OK’;  otherwise,  click  ‘Cancel’.  If  insufficient  data  is  given,  the  user  is  alerted  and 
prompted  to  supply  the  required  data.  Also,  if  the  object  is  outside  of  the  current  grid  display,  a 
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message  box  will  alert  the  user  and  provide  options  to  continue  or  cancel  the  operation.  (Note: 
Monument  objects  cannot  be  created  from  position  messages.) 


Figure  5-63.  Create  Object  from  Position  Message  dialog 
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Figure  5-64.  Object  Position  Data  for  Point  dialog 
Manual  Position  Data  Entry 

A  second  method  of  creating  an  object  is  to  select  the  ‘Create  New  Object’  item  of  the 
‘Edit’  menu  or  by  clicking  the  toolbar  button  shown  in  Figure  5-65.  In  the  Object  data  dialog 
shown  in  Figure  5-66,  select  the  desired  object  type.  Once  the  object  type  is  selected,  the 
associated  object  type  information  controls  are  enabled.  Using  this  method  of  creating  an  object, 
the  Position  Data  is  entered  manually  by  the  user.  All  enabled  text  fields  except  ‘Comments’ 
require  data.  To  create  the  object,  click  ‘Create’;  otherwise,  click  ‘Cancel’.  If  insufficient  data  is 
given,  the  user  is  alerted  and  prompted  to  supply  the  required  data.  Also,  if  the  object  is  outside 
of  the  current  grid  display,  a  message  box  will  alert  the  user  and  provide  options  to  continue  or 
cancel  the  operation. 


Figure  5-65.  Create  New  Object  toolbar  button 
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Figure  5-66.  Object  dialog 


Double-clicking  the  Grid  view 

A  third  method  of  creating  an  object  is  to  double-click  an  empty  location  on  the  grid 
view.  Place  the  pointer  at  the  position  on  the  grid  view  where  the  object  is  to  be  located  and 
double-click.  Upon  double  clicking,  the  Object  dialog  shown  in  Figure  5-66,  is  displayed.  The 
northing  and  easting  positions  corresponding  to  the  location  of  the  pointer  are  displayed  in  the 
Point  1  Northing  and  Easting  edit  boxes  and  the  other  associated  object  type  information  controls 
are  enabled.  The  Point  1  Altitude  is  set  to  the  default  value  specified  for  the  reference  point.  All 
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enabled  text  fields  except  ‘Comments’  require  data.  Using  this  method  of  creating  an  object,  the 
position  data  values  can  be  modified  by  the  user.  To  create  the  object,  click  ‘Create’;  otherwise, 
click  ‘Cancel’.  If  insufficient  data  is  given,  the  user  is  alerted  and  prompted  to  supply  the 
required  data.  (Note:  the  accuracy  of  the  position  data  resulting  from  graphical  operations  may 
not  be  sufficient  for  ground  truth  data.  You  may  want  to  use  these  types  of  operations  for  “gross” 
placement,  and  then  manually  edit  the  data  in  the  object  dialog  to  specify  values  to  greater 
accuracy.) 

Viewing  Object  Information 

The  information  for  an  existing  object  may  be  viewed  and  modified  by  selecting  the 
object  in  several  ways. 

Using  the  Object  List 

To  view  an  object  using  the  object  list,  activate  the  object  list,  following  the  instructions 
in  Section  5. 3.2. 2.4. 1.4,  and  then  double-click  on  the  corresponding  list  item. 

Double-clicking  the  Object  Icon 

To  view  an  object  displayed  on  the  grid  view,  double-click  the  object  icon  using  the  left 
mouse  button. 

Adding  a  Point  from  a  Position  Message  to  an  Existing  Object 

To  add  a  position  point  to  an  existing  object  using  a  position  message,  view  the  position 
message,  following  the  instructions  in  Section  5. 3. 2.2. 4.2. 1.2,  and  then  click  the  ‘Add  to  Existing 
Object’  button.  Upon  making  this  selection,  the  dialog  shown  in  Figure  5-67,  will  display  a  list 
of  all  the  objects  that  can  have  more  than  1  position  point  in  the  survey  session  created  from 
position  messages  from  the  current  surveyor.  To  select  an  object  in  the  dialog,  highlight  the 
object’s  row  and  click  ‘OK’  or  double-click  the  object’s  row.  The  maximum  number  of  position 
data  points  for  an  object  is  four.  If  the  selected  object  already  contains  the  maximum  number  of 
points,  the  user  must  choose  another  object  or  cancel  the  operation.  When  a  valid  object  is 
selected,  the  Object  dialog,  shown  in  Figure  5-64,  is  displayed  showing  the  position  message’s 
northing  and  easting  values  added  to  the  Position  Data  as  the  next  available  point  of  the  existing 
object.  To  confirm  the  addition  of  the  position  data  to  the  existing  object,  click  ‘OK’;  otherwise, 
click  ‘Cancel’. 
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Select  an  object 
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Figure  5-67.  Select  an  Object  dialog 

Modifying  an  Existing  Object 
Using  a  Position  Message 

To  modify  an  existing  object  using  a  position  message,  view  the  position  message, 
following  the  instructions  in  Section  5. 3.2. 2.4. 2. 1.2,  and  then  click  the  ‘Modify  Existing  Object’ 
button.  Upon  making  this  selection,  the  dialog  shown  in  Figure  5-67  will  display  a  list  of  all  the 
objects  in  the  survey  session  created  from  position  messages  from  the  current  surveyor.  To 
select  an  object  in  the  dialog,  highlight  the  object’s  row  and  click  ‘OK’  or  double-click  the 
object’s  row.  The  next  task  is  to  select  the  data  point  to  modify  using  the  dialog  shown  in  Figure 
5-68.  To  select  the  data  point,  highlight  the  data  point’s  row  and  click  OK,  or  double-click  the 
data  point’s  row.  The  Object  dialog  then  opens,  displaying  the  existing  object  with  the  position 
message  values  substituted  into  the  selected  data  point.  To  confirm  the  modification  of  the 
existing  object  with  the  position  message,  click  ‘OK’;  otherwise,  click  ‘Cancel’. 


Select  the  data  point  to  be  modified 
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Figure  5-68.  Data  Point  Selection  dialog 


Using  the  Modify  button 

To  modify  an  existing  object  using  the  ‘Modify’  button,  view  the  object  information 
using  either  of  the  methods  described  in  Section  5. 3. 2.2. 4.2. 3.  Modify  the  desired  value(s) 
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displayed  in  the  Object  dialog.  To  apply  the  modifications,  click  ‘Modify’,  otherwise,  click 
‘Cancel’. 

Moving  the  Object  Icon 

The  position  of  an  existing  object  can  be  changed  by  clicking  on  the  object  icon  with  left 
mouse  button,  holding  the  button  down,  and  moving  the  pointer  to  the  desired  new  location. 
(Note:  the  accuracy  of  the  position  data  resulting  from  graphical  operations  may  not  be  sufficient 
for  ground  truth  data.  You  may  want  to  use  these  types  of  operations  for  “gross”  placement,  and 
then  manually  edit  the  data  in  the  object  dialog  to  specify  values  to  greater  accuracy.) 

Copying  an  Object 

To  copy  an  object,  view  the  object  information  using  either  of  the  methods  described  in 
Section  5. 3.2. 2.4. 2. 3,  then  click  the  ‘Copy’  button.  Modifications  can  be  made  to  any  of  the 
object  information  displayed  in  the  Object  dialog  except  for  the  object  type.  To  save  the  new 
object,  click  ‘Create’,  otherwise,  click  ‘Cancel’. 

Deleting  an  Object 

An  existing  object  may  be  deleted  in  two  ways. 

Using  the  Menu  or  Toolbar  Button 

To  delete  an  object  using  the  menu  or  toolbar  button,  select  the  object  icon  on  the  grid 
view  then  select  the  ‘Delete  Object’  item  of  the  ‘Edit’  menu  or  click  the  toolbar  button  shown  in 
Figure  5-69. 


Figure  5-69.  Delete  Object  toolbar  button 


Using  the  Delete  Button 

To  delete  an  object  using  the  ‘Delete’  button,  display  the  object  information  in  the  Object 
dialog  using  either  of  the  methods  described  in  Section  5. 3.2. 2.4.2. 3,  and  then  click  the  ‘Delete’ 
button. 

Clearing  Alarms  from  a  Lane 

All  alarms  within  a  specified  lane  may  be  cleared  simultaneously  using  the  ‘Clear  Alarms 
from  Lane...’  item  of  the  ‘Edit’  menu.  This  function  does  not  delete  the  alarm  data  from  the 
survey  session,  but  does  clear  the  alarms  from  the  display.  Also,  cleared  alarms  will  not  be 
included  in  subsequent  alarm  files  created  for  the  corresponding  lane.  This  feature  allows  the 
scoring  of  several  exercises  on  a  lane  within  a  single  survey  session  without  deleting  previously 
received  alarms  on  that  lane. 

Verifying  Position  Data  with  a  Monument 

A  Monument  is  a  special  type  of  object  that  allows  the  verification  of  data  received  from 
a  field  surveyor  against  a  well  known,  accurately  measured,  recorded  point.  A  Monument  can  be 
created  only  by  manually  entering  recorded  position  data.  Once  the  Monument  exists,  the  user 
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can  verify  the  data  in  position  messages  sent  from  that  point  by  a  field  surveyor  against  the 
recorded  position.  Clicking  the  ‘Verify’  button  in  the  Monument  data  section  of  the  Object  data 
dialog  opens  the  Verify  Monument  Position  dialog  shown  in  Figure  5-70. 


Figure  5-70.  Verify  Monument  Position  dialog 

This  dialog  displays  the  position  data  recorded  for  the  Monument,  the  position  data  from  a 
position  message  from  the  current  surveyor  at  the  time  the  dialog  was  opened,  and  the  difference 
between  the  two.  Clicking  the  ‘Update’  button  of  this  dialog  will  continuously  update  the 
measured  and  difference  values  using  position  messages  from  the  current  surveyor. 

Communication  Status 

To  view  a  summary  of  the  overall  communication  status,  select  the  ‘View  Connno 
Status’  item  of  the  ‘Commo’  menu.  Selecting  this  item  opens  the  Communications  Status  dialog, 
shown  in  Figure  5-71.  The  information  includes  the  number  of  selected  surveyors,  the  name  of 
the  raw  data  log  file,  and,  for  a  selected  surveyor,  the  current  status  and  number  of 
communications  updates  received.  These  values  are  not  modifiable  from  this  dialog.  In  addition, 
the  corresponding  icon  on  the  grid  display  indicates  the  communications  status  of  an  individual 
field  surveyor.  The  icons  in  Figure  5-51  are  displayed  when  the  current  and  active  surveyors, 
respectively,  are  receiving  updates.  If  the  updates  for  either  cease,  the  icons  in  Figure  5-72  are 
displayed. 
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Communications  Status 
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Log  File  Name: 
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j  OK  1 

Figure  5-71.  Communications  Status  dialog 


Figure  5-72.  Current  and  Active  Surveyor  icons  indicating  No  Communications 


Setting  Options 

The  ‘File’  menu  contains  an  ‘Options’  submenu  that  allows  the  user  to  set  two  features 
that  are  persistent  across  successive  survey  operations.  Each  item  is  set  on  when  the 
corresponding  ‘Options’  submenu  item  includes  a  check  mark.  The  first  option  indicates  whether 
or  not  the  serial  numbers  assigned  to  mines  should  be  included  in  ground  truth  files  created  by 
TMS  Survey.  The  second  option  indicates  whether  or  not  the  user  can  modify  an  object’s 
position  by  dragging  its  icon  within  the  grid  display. 

Survey  Data  GPS  Mode  Checks 

The  data  contained  in  position  messages  received  from  field  surveyors  include  a  GPS 
mode  or  quality  value.  The  ‘Set  GPS  Mode  Checks’  item  of  the  ‘Commo’  menu  opens  the  dialog 
shown  in  Figure  5-73.  This  dialog  allows  the  user  to  turn  on  and  off  two  automatic  actions  based 
on  the  value  of  this  item  in  associated  position  messages. 
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Set  Survey  Data  GPS  Mode  Checks 


[✓  Alert  when  GPS  Mode  changes  from: 
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I*  Prevent  Mine  Creation  when  GPS  Mode  is  below:  FOUR 


"3 


OK 


Cancel 


Figure  5-73.  Set  Survey  Data  GPS  Mode  Checks 

The  first  action  is  to  alert  the  user  when  the  GPS  Mode  value  changes  to  or  goes  below  the 
specified  value.  The  second  action  is  to  prevent  the  creation  of  mine  objects  using  position 
messages  where  the  GPS  Mode  value  is  below  the  specified  value.  (Note:  these  options  are  saved 
between  survey  sessions.) 

Printing  a  Survey  Grid 

To  print  the  current  grid  view,  select  the  ‘Print’  item  of  the  ‘File’  menu  or  click  the 
toolbar  button  shown  in  Figure  5-74. 


Figure  5-74.  Print  toolbar  button 


Printing  a  Ground  Truth  File 

The  contents  of  a  ground  truth  file  can  be  printed  by  selecting  the  ‘Print  Ground  Truth 
File’  item  of  the  ‘File’  menu. 

Saving  Survey  Data 

The  Survey  operation  creates  several  distinct  sets  of  data:  the  survey  session  data;  ground 
truth  files  in  the  WILD  format;  and  alarm  files.  The  following  sections  describe  when  and  how 
these  data  sets  are  saved. 

Creating  Ground  Truth  Files 

Selecting  the  ‘Create  Ground  Truth  File...’  item  of  the  ‘File’  menu  or  the  ‘Create  GT 
File’  toolbar  button,  shown  in  Figure  5-75,  opens  the  Select  File  Data  Contents  dialog,  shown  in 
Figure  5-76.  This  dialog  allows  the  user  to  limit  the  data  written  to  the  file  to  items  in  one  of 
several  categories:  objects  created  from  position  data  from  one  of  any  surveyors  that  are 
currently  active;  objects  created  from  position  data  from  any  source,  including  surveyors  that  are 
or  were  active,  data  entered  manually,  or  from  an  existing  ground  truth  file;  data  pertaining  to  a 
single  lane;  or  all  current  data.  Responding  OK  to  this  dialog  creates  a  ground  truth  file  using 
objects  entered  in  the  system  at  that  time  that  meet  the  limiting  criteria.  Also,  upon  application 
shutdown,  a  dialog  will  be  presented  providing  the  option  of  creating  a  ground  truth  file  from 
object  data  existing  at  that  time.  (Note:  the  default  file  name  extension  for  WILD  format  ground 
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truth  files  used  by  CTMS  is  “.ASC”;  this  file  extension  will  be  applied  by  default  unless  another 
extension  is  provided  as  part  of  the  file  name.) 


B 

Figure  5-75.  Create  GT  File  toolbar  button 


Figure  5-76.  Select  File  Data  Contents  dialog 


Creating  Alarm  Files 

Selecting  the  ‘Create  Alarm  File...’  item  of  the  ‘File’  menu  opens  the  Alarm  Tag  Data 
dialog,  shown  in  Figure  5-77. 
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Alarm  Tag  Data 


I  Contractor/System  ID  [A  -Z):  |  A  1 

Lane: 

L  A 

Run  Number: 

1 

Direction: 

|north 

Include  Mines: 

w 

Include  Clutter: 

w 

OK 

Cancel  | 

Figure  5-77.  Alarm  Tag  Data  dialog 

This  dialog  allows  the  user  to  specify  descriptive  information  that  will  be  used  in  creating  the 
default  file  name,  specifying  the  lane  number  in  which  alarms  are  located,  and  limiting  the 
alarms  to  mine  alarms,  clutter  alarms,  or  both.  Responding  ‘OK’  to  this  dialog  will  then  create 
the  alarm  file  based  on  the  specified  information.  The  ‘Create  Alarm  File...’  menu  item  is 
enabled  only  when  alarms  exist  in  the  current  survey  data.  (Note:  the  default  file  name  extension 
for  alarm  files  used  by  CTMS  is  “.ASC”;  this  file  extension  will  be  applied  by  default  unless 
another  extension  is  provided  as  part  of  the  file  name.) 

Saving  Survey  Session  Data 

Selecting  the  ‘Save  Survey  Data  As...’  menu  item  allows  the  current  survey  session  data 
to  be  saved  to  a  new  file.  Also,  upon  application  shutdown,  a  dialog  will  be  presented  providing 
the  option  of  saving  the  current  survey  session  data  for  the  next  session.  Selecting  this  option 
will  make  the  current  survey  data  available  the  next  time  the  survey  application  is  started.  (Note: 
the  default  file  name  extension  for  survey  session  data  files  is  “.MDB”;  this  file  extension  will  be 
applied  by  default  unless  another  extension  is  provided  as  part  of  the  file  name.) 

Opening  Survey  Data 

Ground  truth  files  and  saved  survey  session  data  files  may  be  used  to  initialize  a  survey 
session.  The  following  sections  describe  opening  ground  truth  files  and  survey  session  data  files. 

Opening  Ground  Truth  Files 

An  existing  ground  truth  file  can  be  opened  using  the  ‘Ground  Truth  File’  item  of  the 
‘Open’  submenu  of  the  ‘File’  menu  or  the  ‘Open  GT  File’  toolbar  button,  shown  in  Figure  5-78. 
Selecting  either  of  these  items  presents  a  file  selection  dialog.  Using  this  dialog,  an  existing 
ground  truth  file  in  the  WILD  format  can  be  selected.  Once  a  file  is  selected,  a  survey  session 
data  set  is  created  from  the  lane  and  object  data  contained  in  the  file.  A  reference  point  is  derived 
from  the  object  data  in  the  file;  however,  as  the  object  data  only  specifies  UTM  easting  and 
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northing  components,  UTM  grid  data  will  have  to  be  provided.  (Note:  the  objects  created  from 
the  data  in  the  file  will  be  identified  as  having  originated  from  a  ground  truth  file.) 


Figure  5-78.  Open  GT  File  toolbar  button 
Opening  Survey  Session  Data 

The  data  from  a  previous  survey  session  can  be  opened  using  the  ‘Saved  Survey  Data’ 
item  of  the  ‘Open’  submenu  of  the  ‘File’  menu.  Selecting  this  item  presents  a  file  selection 
dialog.  Using  this  dialog,  a  previously  saved  survey  session  data  set  can  be  opened. 

Creating  a  New  Survey  Session 

A  new  survey  session  can  be  initiated  using  the  ‘New’  item  of  the  ‘File’  menu  or  the 
‘New  Survey’  toolbar  button  shown  in  Figure  5-79.  After  selecting  this  item,  the  dialog  for 
entering  reference  point  and  lane  identification  data  is  presented. 


Figure  5-79.  New  Survey  toolbar  button 

Exiting  the  Survey  application 

To  exit  the  survey  application,  use  the  ‘Exit’  item  of  the  ‘File’  menu.  If  one  or  more 
objects  has  been  created  or  modifications  have  been  made  that  have  not  been  written  to  a  ground 
truth  file,  a  dialog  will  be  presented  providing  the  option  of  saving  the  objects  to  a  ground  truth 
file.  If  the  user  chooses  to  save  the  objects,  the  Select  File  Data  Contents  dialog  will  open  to 
allow  the  user  to  limit  the  data  in  the  file,  and  a  file  creation  dialog  will  allow  the  user  to  select  a 
file  name  and  a  folder  in  which  the  file  will  be  created.  Also,  upon  exiting  the  survey  application, 
the  user  is  given  the  option  of  saving  the  current  survey  session  data  for  the  next  session.  If  the 
user  selects  to  save  this  data  for  the  next  session,  the  next  time  the  user  starts  a  survey  operation 
he  will  be  prompted  to  use  the  existing  data.  If  the  user  does  not  save  the  data  for  the  next 
session,  he  will  be  given  the  option  of  saving  it  to  an  archive  folder.  Survey  session  data  saved 
to  an  archive  folder  may  be  reopened  within  the  survey  application  using  the  ‘Saved  Survey 
Data’  item  of  the  ‘Open’  submenu  of  the  ‘File’  menu. 

Mine  Interactive  Simulation  Program  (MISP) 

This  section  provides  instructions  in  the  use  and  understanding  of  the  evaluator  interface 
portion  of  MISP.  The  MISP  process  uses  operator-in-the-field  location  information  (x,y,z)  and 
internal  modeling  information  from  the  TMS  Virtual  Mine  Model  (VMM)  database  to  obtain 
various  predictions  and  evaluation  metrics  with  respect  to  the  events  associated  with 
operator/mine  encounters.  MISP  presents  this  information  in  various  dialog  boxes  and  windows. 
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An  evaluator  uses  this  information,  in  the  form  of  visual  cues,  to  direct  and  evaluate  all  trainee 
actions  within  a  real  or  simulated  minefield  environment. 

MISP  Relationships  within  TMS 

TMS  Master  initiates  MISP  process  after  the  evaluator  elects  either  to  run  an  exercise  or 
to  playback  the  data  from  a  previous  exercise.  MISP  initializes  once  the  evaluator  has  selected 
the  ground  truth  data  or  the  exercise  data  file  for  playback.  MISP  obtains  operator  position  and 
detector  response  data  directly  from  the  RGM.  The  processing  of  the  operator  data  by  MISP  is 
controlled  by  the  evaluator  interaction  with  the  TMS  Master  process.  MISP  processing  will  cease 
once  the  evaluator  selects  the  Stop  control  of  TMS  Master,  or,  in  the  case  of  an  exercise 
playback,  when  the  exercise  data  stream  reaches  end-of-file.  MISP  is  active  within  TMS  only 
during  a  live,  playback  or  HLA  exercise  operation. 

Description  of  MISP  Data  Flow/Processing 

Figure  5-80  below  depicts  the  top-level  data  flow  and  processing  associated  with  the 
MISP  process.  Each  Operator  Workstation  is  dedicated  to  a  distinct  exercise.  An  exercise  is 
currently  defined  by  a  particular  operator  (human  or  vehicular)  and  by  an  associated  ground  truth 
file.  The  MISP  process  simply  obtains  and  processes  all  static  and  real-time  data  contained  in  the 
RGM.  For  TMS  Phase  2  this  data  comprises:  (1)  Operator  real-time  component  (foot  or  track 
and  detector)  positions  (x,y,z);  (2)  Operator  real-time  alarm  events;  (3)  Modeling  parameters  for 
mine  detonation  algorithms;  (4)  Modeling  parameters  for  the  blast  effects  (lethal  radius) 
algorithm;  (5)  Modeling  parameters  for  attenuated  pressure  at  the  mine  pressure  plate  as  a 
function  of  burial  depth  and  subsurface  properties;  and  (6)  Baseline  operator  statistical  data  (also 
referred  to  as  average  user  statistics)  used  for  comparisons  and  evaluations.  During  the 
configuration  of  an  exercise,  file  data  residing  in  both  the  VMM  database  file  and  the  ground 
truth  file  is  retrieved  and  placed  in  the  RGM  shared  memory  area  (a  data  structure).  At  runtime, 
an  external  system  obtains  operator  appendage/detector  positions  and  operator  alarms  and  sends 
these  data  items  to  the  DataComms/DSP  component  over  a  digital  link  or  connection. 
DataComms/DSP  processes  and  reduces  this  data  and  places  it  into  RGM;  MISP  then  accesses 
and  processes  this  in  real-time  to  detennine  detonations,  blast  effects,  operator  performance 
evaluations,  and  detector  feedback  responses  to  virtual  mines  (if  any). 
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Figure  5-80.  MISP  data  flow/processing. 

The  Default  MISP  window 

Figure  5-81  below  depicts  the  Default  MISP  window,  which  is  displayed  by  the  TMS 
software  when  an  evaluator  clicks  the  MISP  taskbar  button.  This  button  becomes  visible  when 
the  TMS  Master  launches  (spawns)  the  MISP  process.  This  window  is  relatively  small  in  size.  It 
can  be  repositioned  but  not  resized;  this  is  a  design  feature  intended  to  reduce  potential 
obscuration  of  displayed  TMS  display  information.  Clicking  the  close  (X)  button  located  in  the 
right-hand  portion  of  the  title  bar  minimizes  the  window  but  does  not  tenninate  or  pause  the 
MISP  process.  Clicking  the  TMS  icon,  located  in  the  left-hand  portion  of  the  title  bar,  provides 
access  to  the  process’  system  menu,  which  provides  alternate  methods  with  which  the  evaluator 
can  move  or  close  the  default  window,  and  which  provides  access  to  the  “About”  dialog  for 
MISP. 
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Figure  5-81.  The  Default  MISP  window. 

Description  of  MISP  Window  Functional  Items 

Items  within  the  default  MISP  window  are  intended  mainly  to  provide  operational 
feedback  to  evaluators  of  MISP  status-related  information.  These  include: 

State  text  box 

This  box  contains  the  current  state  of  the  overall  TMS  system  (as  controlled  by  TMS  Master). 
Valid  states  are:  Pre -Exercise,  Exercise,  and  Post-Exercise. 

Initializations  text  box 

This  box  indicates  whether  TMS/MISP  initializations  are  Complete  or  Pending. 

Action  Status  text  box 

This  box  contains  the  current  TMS  Action  Status  as  invoked  by  the  evaluator  using  Master. 
Valid  text  indicators  are:  Run/Play,  Pause,  Stop.  Also,  the  color  MISP  button  on  the  Windows 
taskbar  simultaneously  depicts  this  same  Action  Status  using  the  following  color  scheme  (green- 
Run/Play,  yellow-Pause,  red-Stop). 

Nearest  Mine  -  M  of  N  group  box 

This  box  contains  several  items:  (1)  the  nearest  mine  and  its  coordinates  (Northing,  Easting);  (2) 
Operator  k  coordinates,  where  for  TMS  Phase  2  the  number  k  is  by  definition  k=l  and 
functionality  of  this  list  box  is  disabled  (dimmed);  (3)  the  computed  distance  from  the  nearest 
mine  to  the  nearest  appendage  (foot  or  track)  of  the  selected  field  operator;  (4)  when  the  exercise 
ground  truth  data  includes  virtual  mines,  an  indicator  of  the  horizontal  and  vertical  distance 
between  the  detector  head  and  the  virtual  mine  when  the  head  is  in  close  proximity  to  a  virtual 
mine,  as  well  as  whether  or  not  detector  response  data  is  being  sent  to  the  FIU. 


1-94 


Most  recent  event  (number  &  time) 

This  item  (along  the  lower  window  border)  indicates  the  most  recent  event  and  its  sequence 
number  along  with  a  time  tag.  Defined  events  include  Detonation  alerts,  Operator  Alarms,  and 
mine  Encounters. 

Detonation  indications 

When  the  MISP  detonation  algorithm  determines  that  a  mine  detonation  event  has  occurred,  it 
posts  several  simultaneous  notices  of  this  event.  Two  of  these  notices  involve  the  default  MISP 
window  title  bar  and  the  MISP  taskbar  button.  These  two  items  are  triggered  to  flash  (blink)  for  a 
couple  of  seconds  or  so  after  a  detonation  decision  and  remain  colored  (orange)  thereafter.  At  the 
same  time,  an  acoustic  signal  sounds  (similar  to  the  sound  of  a  revolver  blast).  Refer  to  Sections 
5. 3. 3. 3.1  and  5. 3. 3. 3. 3  for  the  other  detonation  notices. 

The  Expand  button 

Selecting  this  button  opens  the  expanded  MISP  window.  This  new  window  is  arranged  as 
a  Property  Sheet  that  contains  three  pages  accessed  using  the  following  three  tabs:  Exercise  Data 
tab,  Evaluation  tab,  and  Detonations  tab.  Refer  to  the  following  Section  5. 3. 3. 3  for  specific 
details. 

The  Expanded  MISP  window 

Figure  5-82  below  depicts  the  expanded  MISP  window,  which  is  displayed  by  the  TMS 
software  when  an  evaluator  clicks  the  Expand  button  in  the  default  MISP 
window  (see  Section  5. 3. 3.2. 2  above).  Upon  initial  display  of  this  window,  the  page  accessed  by 
the  Exercise  Data  tab  is  selected  by  default.  This  window  can  be  repositioned  but  not  resized. 
Clicking  the  close  (X)  button,  located  in  the  right-hand  portion  of  the  title  bar  of  this  window, 
causes  the  window  to  disappear  but  does  not  terminate  or  pause  the  MISP  process;  in  this  case, 
the  window  in-focus  is  once  again  the  smaller  default  MISP  window. 
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Figure  5-82.  Expanded  MISP  window  /  Exercise  Data  page. 

The  Exercise  Data  page 

Refer  again  to  Figure  5-82  above.  This  page  contains  many  items  of  information  related 
to  the  progress  of  the  exercise. 

State 

Indicates  the  current  state  of  the  overall  TMS  system  (as  controlled  by  Master).  Valid  states  are: 
Pre-Exercise,  Exercise,  and  Post-Exercise. 

Initializations 

Indicates  whether  TMS/MISP  initializations  are  Complete  or  Pending. 

Action  Status 

This  box  lists  the  current  TMS  Action  Status  as  invoked  by  the  evaluator  using  the  TMS  Master. 
Valid  text  indicators  are:  Run/Play,  Pause,  Stop.  Also,  the  MISP  taskbar  button  simultaneously 
depicts  this  same  Action  Status  using  the  following  color  scheme  (green-Run/Play,  yellow- 
Pause,  red-Stop). 

Elapsed  Time 

Uses  the  internal  system  clock  to  record  the  elapsed  time  from  the  beginning  of  the  exercise 
(once  the  evaluator  selects  the  Start  control  of  the  TMS  Master.) 

Operator  group  box 

This  box  contains  operator  ID  selection  (disabled)  plus  operator  and  sensor  locations. 
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ID 

Operator  k  selection;  for  TMS  Phase  2  the  number  k  is  by  definition  k=l  and  functionality  of  this 
list  box  is  disabled  (dimmed). 

Location 

Northing  and  easting  coordinates  (in  meters)  for  the  field  operator’s  left/right  appendages,  and 
the  detector  head  location.  Also,  when  the  exercise  ground  truth  data  includes  virtual  mines,  an 
indicator  of  the  horizontal  and  vertical  distance  between  the  detector  head  and  the  virtual  mine 
when  the  head  is  in  close  proximity  to  a  virtual  mine,  as  well  as  whether  or  not  detector  response 
data  is  being  sent  to  the  FIU. 

The  Mines  (**  indicates  detonation)  list  box 

The  Exercise  Data  page  contains  the  Mines  (**  indicates  detonation)  list  box.  Table  5-1  below 
provides  a  brief  description  of  each  of  the  columns  in  the  list.  During  the  course  of  an  exercise, 
MISP  highlights  the  row  representing  the  undetonated  mine  that  is  currently  nearest  the  operator. 


Table  5-1.  Mines  (**  indicates  detonation). 


Column 

Name 

Description 

ID 

Unique  ID  tag  per  mine;  a  double  asterisk  (**)  will  be 
appended  to  the  tag  if  the  mine  detonates. 

Type 

Typically  the  name  of  the  mine. 

N 

Mine  northing  coordinate  value  in  ground  truth  (in  meters). 

E 

Mine  easting  coordinate  value  in  ground  truth  (in  meters). 

Dist.  (m) 

Distance  in  meters  from  the  mine  to  the  operator  (min.  of 
left/right  foot  distances). 

Foot 

Right  or  left,  whichever  appendage  is  closest  to  the  mine. 

The  Events  button 

Figure  5-83  below  depicts  the  TMS  Exercise  Events  window,  which  is  displayed  by  the  TMS 
software  when  the  evaluator  clicks  the  Events  button  in  the  Exercise  Data  page  of  the  expanded 
MISP  window.  This  window  can  be  repositioned  but  not  resized.  Clicking  the  close  (X)  button, 
located  in  the  right-hand  portion  of  the  title  bar  of  this  window,  causes  the  window  to  disappear 
but  does  not  terminate  or  pause  the  MISP  process;  in  this  case,  the  window  which  comes  in- 
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focus  is  once  again  the  expanded  MISP  window/Exercise  Data  page.  The  Events  button  will  be 
disabled  until  MISP  detects  an  event  during  the  exercise/playback. 


Figure  5-83.  TMS  Exercise  Events  window. 

Description  of  the  Exercise  Events  list  box 

The  TMS  Exercise  Events  window  contains  the  Exercise  Events  list  box.  Table  5-2  below 
provides  a  brief  description  of  each  of  the  columns  in  the  list. 


Table  5-2.  Exercise  Events  List  Contents. 


Column 

Name 

Description 

Number 

Unique  ID  tag  per  exercise  event 

Type 

Information  about  the  event  as  follows: 

•  Encounter  -  ith 

•  Alarm  -  jth 

•  Detonation  -  kth 

where  i,  j,  k  are  running  counters  for  each  of  the  above  event 
types 

Time 

Computer  clock  time  when  event  occurred 

N 

Operator  northing  coordinate  value  (nearest  appendage,  in  meters) 

E 

Operator  easting  coordinate  value  (nearest  appendage,  in  meters) 

Nearest 

Mine 

Numeric  ID  tag  for  nearest  mine 

Distance 

For  this  event,  the  distance  in  meters  to  the  nearest  mine  (min.  of 
left/right  appendage  distances) 
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The  Detonation  Results  window 

Refer  to  the  Mines  list  box  in  the  expanded  MISP  window/Exercise  Data  page  (Figure  5-82). 
Figure  5-84  below  depicts  the  Detonation  Results  window,  which  is  displayed  by  the  TMS 
software  when  an  evaluator  double-clicks  a  row  that  indicates  a  detonated  mine  (as  denoted  by  a 
double  asterisk,  “**”). 


Figure  5-84.  Detonation  Results  Window. 

This  Detonation  Results  window  can  be  repositioned  but  not  resized.  Clicking  the  close  (X) 
button,  located  in  the  right-hand  portion  of  the  title  bar  of  this  window,  causes  the  window  to 
disappear  but  does  not  terminate  or  pause  the  MISP  process;  in  this  case,  the  window  which  is 
in-focus  is  once  again  the  expanded  MISP  window/Exercise  Data  page.  Table  5-3  below 
provides  a  brief  description  of  the  data  displayed  in  this  window. 
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Table  5-3  Detonation  Results  Data. 


Data  Item  Name 

Description 

Detonation  @  xx:yy:zz 

Time  of  detonation 

Lethal  Radius  (m) 

The  lethal  radius  value  (in  meters)  calculated  by 
MISP  at  detonation,  based  on  the  characteristic  data 
for  the  mine  type. 

Lethal  Probability  (%) 

The  lethal  probability  value  (%)  used  to  calculate  the 
lethal  radius. 

Mine  Location  (m) 

The  northing  and  easting  components  of  the  location 
of  the  mine  (in  meters). 

Operator  Location  (m) 

The  northing  and  easting  components  of  the  position 
of  the  left  or  right  appendage  of  the  operator  at  the 
time  of  detonation  (in  meters). 

Operator  Foot  ID 

Left  or  Right,  whichever  appendage  is  closest  to  the 
detonated  mine. 

Halo  Distance  (m) 

The  detonation  halo  distance  (radius  from  the  mine 
pressure  plate  in  meters)  calculated  at  detonation;  a 
value  of  0.0  or  greater  is  listed  for  the  foot  causing  the 
detonation,  a  value  of -1.0  is  listed  for  the  foot  not 
causing  the  detonation. 

Detonation  Threshold  (m) 

The  detonation  threshold  distance  (radius  from  the 
mine  pressure  plate)  within  which  a  detonation  will 
occur. 
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The  Evaluation  Page 

Figure  5-85  below  depicts  the  Evaluation  page  that  is  displayed  by  the  TMS  software 
when  the  evaluator  clicks  the  corresponding  tab.  The  information  MISP  displays  assists  an 
evaluator  in  evaluating  and  directing  the  field  operator  training  session. 


Figure  5-85.  Evaluation  page 

The  Operator  Performance  Statistics  group  box 

There  are  four  items  in  this  box.  Two  of  these  items,  the  number  of  missed  detections  and  the 
false  alarm  rate,  are  TBD/disabled  and,  so,  do  not  display  any  useful  information  for  TMS  Phase 
2.  The  first  of  the  enabled  items,  the  number  of  alarms,  simply  keeps  a  running  sum  of  all  alarms 
for  the  chosen  operator.  Currently,  TMS/MISP  supports  just  one  operator.  The  other  enabled 
item,  the  number  of  mine  encounters,  keeps  a  running  sum  of  all  mines  encountered.  Currently, 
an  encounter  is  defined  using  nearest  distance  measures  and  proximity  thresholds.  The  nearest 
distance  is  measured  from  each  appendage  to  each  mine  centroid,  with  selection  of  the  smaller  of 
these  two  measurements. 

The  Field  vs.  Baseline  Operator  CDF  plots 

The  VMM  Access  database  stores  discrete  data  representing  points  of  the  baseline  operator’s 
Cumulative  Distribution  Function  (CDF)  curve  (also  referred  to  as  Average  User  Statistics).  This 
is  an  experimentally  derived  set  of  points  representing  statistics  against  which  the  field 
operator’s  (trainee’s)  experimental  statistics  are  compared.  Let  Fx(x)  denote  a  CDF  defined  with 
respect  to  the  random  variable  x.  Recall  that  0  <  Fx(x)  <  1  and  that  Fx(xo)  is,  by  definition,  the 
probability  that  x<xo  for  all  extracted  random  variables,  x.  Currently  for  Phase  2,  the  underlying 
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random  variable  is  the  alann  miss  distance,  which  is  defined  as  the  distance  from  the  particular 
alarm  position  to  the  centroid  of  the  nearest  mine.  During  TMS  Setup  mode,  MISP  displays  this 
baseline  curve  in  the  Evaluation  page  by  connecting  every  two  successive  points  in  the  plot  with 
straight  lines.  During  the  course  of  an  exercise,  the  field  operator  (trainee)  generates  alarms  in 
sequence  and  their  miss  distances  fonn  a  discrete  (stair-step)  CDF  curve.  For  every  alarm  update 
this  CDF  curve  changes.  After  every  such  update,  MISP  plots  the  most  recent  discrete  (stair-step) 
CDF  curve.  Plotting  it  in  this  stair-step  fashion  emphasizes  the  discreteness  of  the  data, 
especially  during  the  early  stages  of  an  exercise.  As  an  example,  the  plot  in  Figure  5-83  above 
depicts  the  field  operator  (trainee)  CDF  after  ten  alarms. 

The  Operator  Performance  Meter  (slider  control) 

In  order  to  report  trainee  performance  deviations,  either  positive  or  negative,  relative  to  some 
baseline,  a  computable  metric  must  be  used.  For  Phase  2,  the  manner  of  evaluation  is  as  follows. 
Denote  the  set  of  baseline  operator  miss  distances  as  X=  jxl;  i=l,2,...,Ni}  and  denote  the  set  of 
trainee  miss  distances  as  Y={yj;  j=l,2,...,N2}.  Using  these  sets,  we  obtain  first  order  estimates  of 
their  respective  cumulative  distribution  functions  (CDFs)  using  sorted  (ascending)  sequences,  i. 
e.,  Xsorted  =  {x<i>},  i=l,2,...,Ni  and  Ysorted={y<i>},  i=l,2,...,N2.  Thus,  the  curve  which  passes 
through  the  ordered  set  of  points  Fx={  (x<n>,  1/n);  n=l,2,...,Ni  }  represents  an  estimated  CDF 
derived  from  the  set  X  while  the  curve  which  passes  through  the  ordered  set  of  points  FY  =  {  (y<n>, 
1/n);  n=l,2,...,N2  }  represents  an  estimated  CDF  derived  from  the  set  Y.  As  discussed  in  the 
previous  section,  TMS/MISP  displays  both  of  these  curves,  Fx  and  FY,  in  real  time  within  the 
expanded  MISP  window/Evaluation  tab.  A  simple  metric,  used  to  measure  the  difference 
between  Fx  and  FY,  is  the  integrated  difference  (nonnalized  via  a  sigmoidal  map).  It  is: 

mdiff  =  S{ZI  \  Where  Z/  =  Operator  ~  1  Baseline  ) 

s(*)  is  monotonically  increasing,  s(-co)  =  -1,  and  s(+oo)  =  +1.  Currently,  the  sigmoidal  map  s(-)  is 
s(z)  =  arctan(k-z).  The  factor  k  is  an  expansion  or  compression  parameter  intended  to  improve 
the  intuitive  feel  of  the  final  performance  value.  Presently,  k=l. 

For  continuous  curves, 


I^„„,=C~FA’-)dr 
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for  which  Rmax  =  Max(ri,  r2)  with  ri=smallest  r  such  that  Fx(r)=l  and  r2=smallest  r  such  that 
FY(r)=l.  For  our  case  of  discretely  defined  curves, 


N,- 1 


1  Baseline 


A  +  z— -ta+b-O  with  A  = 

n= 1  N  l 


N,- 1 


^ Operator  ^2  Z— iy<n+i>-y<n>)  with  A 


n= 1  N2 


\y<N2> 

X<Nl>  fi 

x<no  <  y<N2> 

0, 

x<Nl>  *  y<N2 

■  ~y<N2>\ 

y<N2> <  x<n1> 

1°, 

y<N2>  ^  x<N , 

Due  to  the  sigmoidal  normalization,  the  computed  value  for  mdiff  is  always  between  -1  and  +1. 
TMS/MISP  displays  the  outcome  of  this  computation  using  the  position  of  a  slider  control  in  the 
Operator  Performance  Meter  Group  box  on  the  Evaluation  page. 

The  Operator  group  box 

Items  in  this  box  deal  with  the  field  operator  alarm  data  used  in  the  plotted  display. 

The  Operator  N  list  box 

For  TMS  Phase  2,  the  number  N  is  by  definition  N=1  and  so  the  functionality  of  this  list  box  is 
currently  disabled  (dimmed).  If  enabled  at  some  point,  selection  of  Operator  N  will  display  data 
in  the  entire  window  exclusively  for  the  Nth  field  operator. 


The  Show  Alarms  button 

This  button,  when  clicked,  presents  a  display  of  all  field  operator  alarms  up  to  the  present  time. 
Figure  5-86  below  depicts  the  Alarms  window  per  such  a  request.  Clicking  the  close  (X)  button, 
located  in  the  right-hand  portion  of  the  title  bar  of  this  window,  causes  the  window  to  disappear 
but  does  not  terminate  or  pause  the  MISP  process;  in  this  case,  the  window  in-focus  is  once  again 
the  expanded  MISP  window/Evaluation  page. 


Figure  5-86.  Alarms  data  window. 

The  Alarms  list  box 

The  Alarms  window  contains  the  Alarms  list  box.  Table  5-4  below  provides  a  brief  description 
of  each  of  the  columns  in  the  list. 
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Table  5-4.  Alarms  List  Contents. 


Column  Name 

Description 

Number 

Unique  ID  tag  per  alarm. 

Time 

Computer  clock  time  when  alarm  notification  was  placed. 

N 

Alarm  location  northing  coordinate  value  (in  meters). 

E 

Alarm  location  easting  coordinate  value  (in  meters). 

Nearest  Mine 

Numeric  ID  tag  for  nearest  mine. 

Distance  (m) 

Distance  from  alarm  to  nearest  mine  (in  meters). 

The  Detonations  Page 

Figure  5-87  below  depicts  the  Detonations  page  that  is  displayed  by  the  TMS/MISP 
software  when  an  evaluator  clicks  the  corresponding  tab  in  the  expanded  MISP  window.  The 
information  MISP  displays  relates  to  detonated  mines  and  predicted  blast  effects,  in  particular 
the  lethal  radius  estimate. 


Figure  5-87.  Detonations  Page  in  the  Expanded  MISP  Window. 

The  Operator  group  box  and  Operator  N  list  box 

For  TMS  Phase  2,  the  number  N  is  by  definition  N=1  and  so  the  functionality  of  this  list  box  is 
currently  disabled  (dimmed).  If  enabled  at  some  point,  selection  of  Operator  N  will  display 
detonation  data  on  this  page  exclusively  for  the  Nth  field  operator. 
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Description  of  Detonated  Mines  group/list  box 

Table  5-5  below  provides  a  brief  description  of  each  of  the  columns  in  the  list.  When  an 
evaluator  double-clicks  a  particular  row  in  the  list,  the  Adjust  Lethal  Probability  window  appears 
(see  next  section). 


Table  5-5  Detonated  Mines  List  Contents. 


Column 

Name 

Description 

Order 

Sequential  numbering  tag  per  detonation  per  operator. 

ID  -  Type 

Mine  ID  and  Type  (name);  for  example, 

“2-M14”  denotes  ID=2  and  Type=M14 

N 

Mine  location  northing  coordinate  value  (in  meters). 

E 

Mine  location  easting  coordinate  value  (in  meters). 

Leth.  Rad. 

Computed  lethal  radius  value  (in  meters). 

Leth.  Prob. 

Requested  lethal  probability  (%),  defaulted  from  VMM  during 
setup  or  reentered  at  runtime  (by  double  clicking  a  row  in  the 
list). 

The  Adjust  Lethal  Probability  window 

Figure  5-88  below  depicts  the  Adjust  Lethal  Probability  window.  To  open  this  window  the 
evaluator  must  initially  be  in  the  expanded  MISP  window/Detonations  page  and  double  click  the 
row  in  the  Detonated  Mines  list  box  that  corresponds  to  a  mine  of  interest.  Clicking  the  close  (X) 
button,  located  in  the  right-hand  portion  of  the  title  bar  of  this  window,  causes  the  window  to 
disappear  but  does  not  tenninate  or  pause  the  MISP  process;  in  this  case,  the  window  in-focus  is 
once  again  the  expanded  MISP  window/Detonations  page. 
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Figure  5-88.  The  Adjust  Lethal  Probability  window. 

The  Mine  and  Location  text  boxes 

The  display-only  information  in  these  boxes  is  associated  with  the  chosen  mine  (its  ID,  type,  and 
location). 

Enter  the  New  Lethal  Probability 

The  evaluator  enters  a  new  value  for  the  lethal  probability  in  integral  percentage  units  (1  -  100) 
directly  or  via  the  spin  button  control. 

The  Apply  button 

Clicking  this  button  forces  a  new  computation  of  the  MISP  Phase  2  blast  effects  algorithm  using 
the  requested  lethal  probability  value  as  described  above  and  the  modeling  parameters  contained 
in  the  VMM  database.  The  output  of  this  algorithm  is  the  lethal  radius.  This  output  may  be 
observed  in  the  entry  for  the  selected  mine  in  the  Detonated  Mines  list  box  on  the  Detonations 
page  of  the  expanded  MISP  window.  It  may  require  some  repositioning  of  the  Adjust  Lethal 
Probability  window  in  order  to  observe  these  numbers.  Note  also  that  the  evaluator  need  not 
reenter  a  new  lethal  probability  value  before  clicking  the  Apply  button;  a  new  lethal  radius  value 
will  be  calculated  each  time  using  the  probabilistic  parameters  in  effect. 

The  Reset  button 

This  button  resets  the  lethal  probability  to  the  default  value  listed  in  the  VMM  and  resets  the 
lethal  radius  to  the  value  initially  computed  upon  mine  detonation. 

The  OK  button 

This  button  performs  the  same  function  as  the  Apply  button,  but  closes  the  window  after 
performing  the  lethal  radius  calculation. 

The  Cancel  button 

This  button  closes  the  window  without  performing  the  lethal  radius  calculation. 

CTMSHost 

The  CTMSHost  application  provides  the  interface  between  the  TMS  Master  and  the 
Countermine  Test  Management  System  (CTMS).  In  TMS  Phase  II,  CTMS  executes  on  a 
separate  computer  from  other  TMS  evaluation  and  simulation  components.  The  Master  and 
CTMSHost  interact  through  the  exchange  of  messages  over  a  dedicated  Ethernet  connection. 
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Local  to  the  CTMS  workstation,  CTMSHost  initializes  RGM  compliant  with  the  TMS  Phase  1 
application  programming  interface  (API)  for  data  and  event  communications  between  itself  and 
CTMS,  invokes  the  CTMS  Reports  Manager  for  post-exercise  analysis,  and  provides  the 
mechanism  for  configuring  CTMS  for  Exercise  and  Playback  operations. 

Startup 

At  startup,  the  CTMSHost  and  CTMS  windows  appear  as  shown  in  Figure  5-89.  Figure 
5-90shows  the  CTMSHost  underlying  startup  menus.  The  Exit  menu  terminates  both 
CTMSHost  and  CTMS  and  sends  a  notification  message  to  the  Master.  The  ‘Analysis’  item  of 
the  ‘Operations’  menu  allows  the  evaluator  to  invoke  the  CTMS  Reports  Manager  for  post¬ 
operation  analysis  of  exercise  results. 


Figure  5-89  CTMSHost  and  CTMS  main  windows 
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Figure  5-90.  Startup  menus 


Operations 

Exercise 

To  execute  an  Exercise  operation  on  the  CTMS  workstation  simultaneously  with  TMS 
Master,  the  exercise  must  first  be  initiated  on  the  Master  by  following  the  procedures  in  Section 
5.3.2.2.I.  To  load  the  ground  truth  file  in  CTMS,  select  the  ‘New  Test  Importing’  item  of  the 
CTMS  ‘File’  menu.  In  the  ‘Target,  Lane  and  Landmark  Files’  dialog,  select  the  appropriate 
ground  truth  file  and  click  ‘Open’.  In  the  next  dialog,  ‘Characteristics  for  New  Test  Data’,  enter 
an  ID  for  the  exercise  and  click  ‘OK’.  The  dialog’s  additional  fields  are  not  required.  After  the 
ground  truth  file  is  loaded,  the  status  bar  message  turns  to  ‘Ready’  and  the  ‘Start’  button  and  the 
‘Start’  item  on  the  ‘Actions’  menu  are  enabled.  (Note:  to  ensure  that  alarms  received  during  an 
exercise  are  saved  to  the  CTMS  database  for  inclusion  in  post-operations  analysis,  the  ‘Save 
Automatically’  item  of  the  CTMS  ‘File’  menu  should  not  be  selected  -  selection  is  indicated  by  a 
check  mark  by  the  item  in  the  menu.) 

Figure  5-91  shows  the  CTMS  workstation  applications  configured  for  an  exercise.  The 
first  pane  on  the  CTMSHost  main  window  status  bar  displays  the  operation  type.  The  second 
pane  displays  the  ground  truth  filename  and  the  third  pane  is  the  exercise  name.  The  icons  on 
the  grid  display  depict  the  objects  in  the  ground  truth  file. 
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Figure  5-91.  Exercise  operation 


Exercise  Functions 

The  following  sections  describe  the  application  controls  and  options  provided  during  an 
exercise  operation. 

Start 

Refer  to  Section  5.3.2.2.1.1.13.  Note:  The  exercise  can  be  started  in  the  TMS  Master  or 
CTMSHost. 

Pause 

Refer  to  Section  5.3.2.2.1.1.14.  Note:  The  exercise  can  be  paused  in  the  TMS  Master  or 
CTMSHost. 

Stop 

Refer  to  Section  5.3.2.2.1.1.15.  Note:  The  exercise  can  be  stopped  in  the  TMS  Master  or 
CTMSHost. 

Stop  Exercise  in  CTMS 

CTMS  may  experience  a  delay  in  reading  and  displaying  exercise  data,  hence  after  an 
exercise  is  stopped,  a  dialog  is  launched  prompting  the  evaluator  to  stop  the  exercise  in  CTMS  or 
continue.  To  eliminate  the  ‘Stop  Exercise  in  CTMS’  dialog,  check  the  ‘Apply  to  subsequent 


1-109 


exercises’  option  and  click  ‘OK’.  Checking  the  ‘Stop  Exercise  in  CTMS’  item  of  the  ‘Options’ 
submenu  of  the  ‘File’  menu  also  eliminates  the  dialog. 

Saving  Alarms  in  CTMS 

For  alarms  reported  during  an  exercise  to  be  saved  in  the  CTMS  database  for  inclusion  in 
analysis  provided  by  the  CTMS  Reports  Manager,  they  must  be  saved  manually  in  CTMS.  To 
save  the  alarms,  select  the  ‘Save  Now’  item  of  the  ‘File’  menu  in  CTMS. 

After  an  exercise  is  stopped,  a  dialog  box  is  launched  to  remind  the  evaluator  that  alarms 
have  to  be  saved  manually  in  CTMS.  To  eliminate  this  dialog,  check  the  ‘Apply  to  subsequent 
exercises’  option  and  click  ‘OK’.  Checking  the  ‘Display  'Save  Alarms'  message’  item  of  the 
‘Options’  submenu  of  the  ‘File’  menu  also  eliminates  the  dialog. 

Playback 

To  execute  a  playback  exercise  from  CTMSHost,  the  exercise  must  first  be  initiated  on 
the  Master  by  following  the  procedures  in  Section  5. 3.2. 2.2.  CTMS  does  differentiate  between  a 
live  exercise  and  playback  exercise. 

Figure  5-92  shows  the  CTMS  workstation  applications  after  a  playback  exercise  is 
configured.  The  CTMSHost  status  bar  pane  contents  are  identical  to  those  during  an  Exercise 
operation.  The  first  pane  on  the  status  bar  displays  the  operation.  The  second  pane  displays  the 
ground  truth  filename  and  the  third  pane  is  the  exercise  name. 
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Figure  5-92.  Playback  operation 
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Playback  Functions 

The  following  sections  describe  the  application  controls  and  options  provided  during  a 
playback  operation. 

Start 

Refer  to  Section  5.3.2.2.1.1.13.  Note:  Playback  can  be  started  in  the  TMS  Master  or 
CTMSHost. 

Pause 

Refer  to  Section  5.3.2.2.1.1.14.  Note:  Playback  can  be  paused  in  the  TMS  Master  or 
CTMSHost. 

Stop 

Refer  to  Section  5.3.2.2.1.1.15.  Note:  Playback  can  be  stopped  in  the  TMS  Master  or 
CTMSHost. 

Fast  Forward 

Refer  to  Section  5.3.2.2.2.1.15.  Note:  Fast  forward  can  be  manipulated  in  the  TMS 
Master  or  CTMSHost. 

Stop  Exercise  in  CTMS 

Refer  to  Section  5. 3. 4.2. 1.1. 4. 

Saving  Alarms  in  CTMS 

Refer  to  Section  5. 3. 4.2. 1.1. 5. 

Analysis 

Using  the  ‘File’  menu,  select  ‘Operations’— C Analysis’— CCTMS  Reports  Module’,  to 
initiate  Analysis  under  CTMSHost.  The  CTMS  Reports  Manager  invokes  Microsoft  Access,  as 
shown  in  Figure  5-93. 

Refer  to  the  CTMS  User’s  Guide  for  instructions  on  displaying  and  printing  Report 
Module  graphs  and  reports.  The  ID  entered  in  the  dialog,  ‘Characteristics  for  New  Test  Data’, 
referred  to  in  Section  5. 3. 4.2.1,  is  the  ID  that  is  entered  in  the  ‘Test  Id’  field  of  the  Test  Reports 
Manager. 
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Figure  5-93.  CTMS  Reports  Manager 


Field  Instrumentation 

The  TMS  field  instrumentation  comprises  hardware  and  software  used  to  acquire  and 
send  data  associated  with  mine  detector  operations  to  TMS  DataComms/DSP  and  to  receive 
commands  and  data  from  TMS  DataComms/DSP.  The  field  instrumentation  is  considered  an 
embedded  system  and  functions  without  any  display  device  during  normal  operations.  In  the 
following  sections,  figures  depicting  screen  shots  are  used  to  illustrate  the  processing  within  the 
field  instrumentation  and  would  not  be  visible  or  accessible  to  a  mine  detector  operator  during 
normal  use. 


Operator  Client  -  Data  Acquisition  and  Communications  Application 

The  Operator  Client  application  (“OpCli.exe”)  acquires  mine  detector  and  position  data 
and  sends  it  to  DataComms/DSP.  OpCli.exe  retrieves  XYZ  position  by  using  the  GetPosition 
method  exposed  by  the  Arc  Second,  Inc.  3DiWorkbench  application.  Mine  detector  data  is 
retrieved  by  either  reading  a  data  stream  from  the  RS-232  port  or  from  an  Analog-to-Digital 
converter,  depending  upon  the  type  of  detector.  Note:  Currently  TMS  has  been  tested  with  2 
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mine  detectors:  the  F-3  and  AN/PSS-12.  The  F-3  outputs  data  via  an  RS-232  port.  The 
AN/PSS-12  outputs  analog  data.  SRC  modified  the  AN/PSS-12  such  that  the  analog  data  is 
input  into  an  Analog  to  Digital  converter  whose  form  factor  is  PCMCIA.  OpCli.exe  then  reads 
the  values  from  the  A-to-D  converter.  As  more  mine  detectors  are  integrated  with  TMS,  other 
data  acquisition  methodologies  may  be  implemented. 

OpCluexe  Command-Line  Parameters 

OpCli.exe  makes  use  of  Windows  initialization  files  (“.INI”  extension)  that  contain  sections  and 
key  names  (parameter  settings)  to  configure  an  instance  of  OpCli.exe  such  that  its  runtime 
operation  is  characterized  by  the  initialization  file  parameters.  OpCli.exe  uses  two  files: 
Operator.ini  and  TMS_Comm.ini.  Operator.ini  contains  parameters  that  pertain  to  the  operator 
software  functionality  only.  TMS_Comm.ini  contains  parameters  that  apply  to  both 
DataComms/DSP  and  OpCli.exe.  Both  of  these  files  reside  in  the  Windows  root  directory 
(C:\Windows.)  Given  the  command-line 

C:\TMS_Home\TMS_Bi  n  \  0  p  C I  i  .  exe  OPERATOR  6 

OpCli.exe  will  use  the  key  name  parameter  settings  in  the  section  OPERATOR  6,  in  both  the 
Operator.ini  and  TMS_Comm.ini  files  (see  Sections  5. 3. 5. 1.2  and  5. 3. 5. 1.3  for  detailed 
descriptions  of  the  “.INI”  files)  to  define  its  runtime  operability. 

Operator.ini 

Table  5-6  describes  each  of  the  key  name  entries  in  the  Operator.ini  file  and  shows  all 
possible  values  for  each  key  name. 

Note:  The  contents  of  the  .INI  files  are  listed  for  the  edification  of  the  reader.  The  parameter 
values  in  these  files  are  configured  to  provide  optimal  system  performance.  Modification 
of  these  files  is  NOT  recommended. 
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Table  5-6.  Operator.ini  Key  name  Descriptions 


Item 

Values 

Description 

[OPERATOR  61 

NA 

Section  Name 

CommPort 

1  -  10 

Com  Port  #  used  for  remote  communications 
purposes  (Default:  None) 

CommBaudRate 

300 

1200 

2400 

4800 

9600 

19200 

38400 

57600 

115200 

Baud  rate  used  for  remote  communications 
purposes  (Default:  None) 

MineDetComPort 

1  -  10 

Com  Port  number  when  communicating  with 
mine  detector  (Default:  5) 

MineDetBaudRate 

300 

1200 

2400 

4800 

9600 

19200 

38400 

57600 

115200 

Baud  rate  used  when  communicating  with 
the  mine  detector  (Default:  9600) 

MineDetType 

ANPSS12 

F1A4 

F3 

Mine  detector  type  (this  key  name  is  ignored 
if  the  MineDetAutoDetect  key  name  is  set  to 
1)  (Default:  ANPSS 12) 

SensorUpdateRate 

10 

20 

30 

40 

50 

60 

70 

80 

90 

100 

The  sensor  data  acquisition  update  rate  in 
hertz  (Default:  20) 

PositionUpdateRate 

1 

2 

3 

4 

5 

6 

The  position  data  acquisition  update  rate  in 
hertz  (Default:  10) 
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Item 

Values 

Description 

7 

8 

9 

10 

PositionSysType 

Time  Domain 
Arc  Second 

The  positioning  system  type  (Default: 
ArcSecond) 

MineDetAutoDetect 

0-Off 

1  -  On 

Flag  detennining  whether  or  not  to  use  the 
auto-configure  feature  to  automatically 
detect  what  type  of  mine  detector  is 
connected  to  the  instrumentation  package 
(do  not  set  this  the  auto  detect  key  name  to  1 
when  testing  with  the  Arc  Second  position 
instrumentation  using  the  same  port  as  the 
F3;  when  the  port  is  initialized  to  attempt  to 
read  F3  data,  a  fatal  error  occurs)  (Default: 
1) 

SimSensorFeedbacklnput 

0-Off 

1  -  On 

Flag  detennining  whether  or  not  to  manifest 
the  Virtual  Mine  Modal  feedback  as  a 
function  of  PC  speaker  (using  the  Beep 
API);  or  reading  from  the  mine  detector  after 
the  value  has  been  injected  into  the  mine 
detector  (Default:  0) 

AlarmKeyPressInterval 

2- 2  seconds 

3- 3  seconds 

4- 4  seconds 

5- 5  seconds 

Flag  determining  how  long  to  wait  for  a 
second  key  press  after  an  initial  alarm 
enunciator  key  has  been  pressed  (Default:  3) 

TMS_  Comm,  ini 

Table  5-7  describes  each  of  the  key  name  entries  in  the  TMS_Comm.ini  file  and  shows 
all  possible  values  for  each  key  name. 
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Table  5-7.  TMS_Comm.ini  Key  name  Descriptions 


Item 

Values 

Description 

NetType 

ETHERNET 

SERIAL 

PI  SERIAL 

SURVEY  INSTR  AS 
SURVEY  SERIAL 

The  network  type  (Default: 

ETHERNET) 

Host  ID 

Valid  IP  Address 

Host  (DataComrns/DSP)  IP  address 
(Default:  NA) 

TransportType 

UDP 

TCP 

Transport  type  (Default:  UDP) 

3DiWorkbench  -  Angle  to  XYZ  Conversion 


Figure  5-94.  3DiWorkbench 
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The  Arc  Second,  Inc.  3DiWorkbench  Windows  Application  acquires  angles  and  converts 
them  to  XYZ  coordinates.  3DiWorkbench  is  a  position  data  server.  That  is,  it  exposes  methods 
and  properties  that  are  used  by  Operator  Client  to  extract  XYZ  position  data  from  memory. 
Figure  5-94  shows  the  3DiWorkbench  application  main  window. 


Operator  Client  and  3DiWorkbench  Execution 

To  start  sensor  and  position  data  acquisition  on  the  FIU,  3DiWorkbench  and  Operator 
Client  must  be  executed  respectively.  3DiWorkbench  is  executed  first  to  connect  to  the  PCEs  to 
begin  collecting  angle  data  and  converting  it  to  position  data  (X,Y,Z).  Next,  Operator  Client  is 
executed  with  the  appropriate  command  line  parameter.  The  procedure  to  begin  data  acquisition 
and  transmission  on  the  FIU  is  as  follows: 

Note:  The  following  procedure  assumes  the  following: 

•  A  Wireless  Ethernet  connection  to  the  FIU  is  available 

•  3DiWorkbench  calibration  has  been  completed  (see  Appendix  A) 

•  The  optical  detectors  and  the  metal  detector  are  properly  connected  to  the  FIU  (the  metal 
detector  should  be  on) 

1 .  Launch  PCAnywhere  on  the  appropriate  workstation  and  connect  to  the  FIU  of  interest  (See 
Figure  5-95). 


Figure  5-93.  PCAnywhere  window  as  displayed  on  the  TMS  Workstation 
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2.  On  the  FIU,  launch  3DiWorkbench  (See  figure  5-96). 


FIU3  -  Symantec  pcAnywhere  Remote 


File  Edit  Task  Actions  Help 


0 

0 


Remote  Control 


Figure  5-96.  3DiWorkbench  Initial  Display  as  displayed  in  PC  anywhere 


3.  Within  3DiWorkbench,  from  the  File  menu,  select  Open.  From  the  resulting  file  selection 
dialog,  select  the  file  \TMS_Home\TMS_Bin\FIU.3Di  (See  Figure  5-97). 
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Figure  5-97.  Opening  the  FIU.3DI  File  in  3DiWorkbench 

4.  From  the  3Di  Tree  view  (left  pane)  display,  click  on  the  Input  Device  branch.  In  the 

resulting  Input  Devices  Property  Section  (right  upper  pane),  under  the  Properties  tab,  click 
the  Start  All  button.  Non-zero  position  values  should  appear  in  the  AsiServerSinglePoint 
dialogs  entitled  “0”,  “1”,  “2”  and  “3”  (See  Figure  5-98).  Minimize  the  3DiWorkbench 
window. 
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FIU3  -  Symantec  pcAnywhere  Remote 


File  Edit  Task  Actions  Help 


File  Transfer 
Q  Command  Queue 
a  Show  Chat 
x  End  Session 


Remote  Control 

9  — 

Screen  Scaling 
^  View/Edit  Online  Options 
1^  T  ransfer  Clipboard 

S tart/S top  Session  Recording 
Save  Current  Screen 
3*  Send  Ctrl+Alt+Del 
@  Restart  Host  Computer 


5 


Details 

$3  FIU3 

169.254.39.10 

TCP/IP 

(jp  Encryption  Level:  None 
Windows  XP 
Connected 

T  ime  of  S  ession:  00: 06: 37 
Send:  20  KB 

j2 Receiveii298KB^^^^ 


* 


3DIWorkbench  •  FIU.3di 


Remote  Control 

m 

File  Transmitters  Server  Setup  Too[s  Help 

□  a?y 


Conductor 
+  Configurations 
+  Detectors 

-  Input  Devices 
®  II 

S  12 
E  13 
[+  14 

-  Servers 
B0 
®  1 
B  2 
E  3 

Connections 
l±  Display 
Settings 


Properties  Messages 


Input  Devices: 


Add  Remove  Rename  Start  All  I  Stop  All 


y 


y 


E  * 


X:  1 1.941 23938 
Y:  1 3.2751 8707 
z  -1  05076223 
M:  0  00778410 


1 2. 28444766 


1.05669726 
M:|0. 007301 96 


* 

'  *1 


1 2. 92620223 


•0.26220483 
M:  [0.00395217 


•0.63546917 
1 0.0051 6571 


/Start  O  C:\Documents  and  .. .  &  3DIWorkbench  -  FI... 


Figure  5-98.  3DiWorkbench  Calculating  Positions  From  Angles 


5.  On  the  FIU,  launch  Operator  Client  by  opening  a  console  window,  navigating  to  the 
\TMS_Home\TMS_BIN  directory  and  entering  the  following  command  line: 

C:\TMS_Home\TMS_Bi  n\opcl  i  .  exe  OPERATOR^ 

Note:  The  OPERATOR  3  parameter  is  used  as  an  example.  The  actual  parameter  used  will 
depend  on  the  configuration  of  the  FIU  of  interest  (See  section  5.3.5. 1 . 1  for  more  information 
on  using  Operator  Client  command  line  parameters). 

6.  Close  all  open  windows  on  the  FIU  except  3DiWorkbench  (which  should  currently  be 
minimized)  and  Operator  Client.  Bring  Operator  Client  window  to  the  foreground,  maximize 
it  and  ensure  that  it  is  the  active  window.  This  is  necessary  to  ensure  that  Operator  Client  is 
the  foreground  application  to  process  input  events  corresponding  to  alarm  notifications 
generated  by  the  operator  (See  Figure  5-99). 
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FIU3  -  Symantec  pcAnywhere  Remote 


File  Edit  Task  Actions  Help 


■£}v  File  Transfer 
Q  Command  Queue 
a  Show  Chat 
x  End  Session 


Remote  Control 

9  — 

Screen  Scaling 
^  View/Edit  Online  Options 
1^  T  ransfer  Clipboard 

S tart/S top  Session  Recording 
Save  Current  Screen 
3*  Send  Ctrl+Alt+Del 
@  Restart  Host  Computer 
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Details 

$3  FIU3 

169.254.39.10 

TCP/IP 

(Jp* 1 2 3 4 5  Encryption  Level:  None 
Windows  XP 
Connected 

Time  of  Session:  00:24:12 
Send:  24  KB 
Receive:  7349  KB 


* 


Remote  Control 


cv  Command  Prompt  -  opcli  OPERATOR_3 

(Local  Information: 


01 


Host  Name: 

HP  in  use: 

Sleep: 

Sensor  Update  Rate: 


f  iu3 

169.254.39.10 
8  ms 

20  Hz  <50  ns) 


[Position  Update  Rate:  10  Hz  <100  ns) 


[Remote  Information: 


Host  Name: 

IIP 


Unknown 

169.254.39.3 


press  "x"  to  exit: 

FindPositionAppProc :  "3DI Workbench"  found...  OK  to  send  position  data 
IfiutoDetectProc :  Found  an  "AN/PSS-12"  mine  detector... 


m 

► 

|  t  Start  O  C:\Documents  and  ... 

|  |  3DI Workbench  -  FIU. . .  1 1 Q9  Command  Prompt . . . 

Figure  5-99.  Operator  Client  console  window  as  displayed  in  PC  Anywhere 

The  FIU  should  now  be  transmitting  sensor  and  position  data  to  DC/DSP.  The  PCAnywhere 
session  at  the  TMS  Workstation  should  be  terminated  at  this  point. 

FIU  Shutdown 

Since  the  FIU  is  a  headless  platform,  the  shutdown  procedure  requires  the  use  of 
PCAnywhere.  The  procedure  for  shutting  down  the  FIU  is  as  follows: 

1.  Launch  PCAnywhere  on  the  TMS  Workstation  and  connect  to  the  FIU  (See  Figure  5-95). 

2.  Click  on  the  Operator  Client  window  and  enter  an  “x”.  Operator  Client  will  kill  the 
3DiWorkbench  process  and  then  terminate  itself. 

3.  Click  the  Windows  XP  Start  button. 

4.  Click  the  Turn  Off  Computer  item. 

5.  Click  the  Turn  Off  button  (See  Figure  5-100). 
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Figure  5-100.  Windows  XP  Shutdown  Window 

After  some  hard  disk  activity,  the  red  LED  indicator  located  on  the  front  panel  of  the  FIU  will  go 
out.  This  is  an  indication  that  the  FIU  has  been  successfully  shut  down. 

Related  Processing 

No  additional  processing  not  described  herein  is  performed  by  TMS. 

Data  Backup 

No  TMS  component  has  a  requirement  for  an  integrated  or  automated  capability 
specifically  relating  to  data  backup.  Any  data  file  used  or  generated  by  any  TMS  component  may 
be  transferred  to  an  appropriate  available  storage  device  by  the  user  via  the  standard  Windows 
file  transfer  mechanisms. 

Recovery  from  Errors,  Malfunctions,  and  Emergencies 

No  specific  procedures  for  restart  or  recovery  from  errors  or  malfunctions  during  system 
processing  are  defined  for  TMS. 

Messages 

No  TMS  application  presents  to  the  user  any  coded  messages  in  response  to  abnormal 
processing. 

Quick  Reference  Guide 

This  paragraph  has  been  tailored  out. 
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Notes 

Acronyms 


Acronym 

Meaning 

BRTRC 

Baum-Romstedt  Technology  Research  Corporation 

CD 

Compact  disc 

CDF 

Cumulative  Distribution  Function 

CDR 

Critical  Design  Review 

CDRL 

Contract  Data  Requirements  List 

CM 

Countermine 

COR 

Central  Outdoor  Router 

CSCI 

Computer  Software  Configuration  Item 

CTMS 

Countennine  Test  Management  System 

DAQ 

Data  Acquisition 

DATA  COMMS/DSP 

Data  Communications/Digital  Signal  Processing 

DIS/HLA 

Distributed  Interactive  Simulation/High  Level  Architecture 

DMSO 

Defense  Modeling  and  Simulation  Organization 

FIU 

Field  Instrumentation  Unit 

FOM 

Federation  Object  Model 

GPS 

Global  Positioning  System 

GSTAMIDS 

Ground  Standoff  Mine  Detection  System 

GUI 

Graphical  User  Interface 

HLA 

High  Level  Architecture 

HSTAMIDS 

Handheld  Standoff  Mine  Detection  System 

HWCI 

Hardware  Configuration  Item 

IP 

Internet  Protocol 

JT&E 

Joint  Test  and  Evaluation 

MISP 

Mine  Interaction  Simulation  Program 

MS 

Microsoft  Corporation 

NVESD 

Night  Vision  and  Electronic  Sensors  Directorate 

OITL 

Operator-In-The-Loop 
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Acronym 

Meaning 

PC 

Personal  Computer 

PCE 

Position  Calculation  Engine 

PDR 

Preliminary  Design  Review 

PLT 

Position,  Location  and  Tracking 

PM-CCS 

Program  Manager  -  Close  Combat  Systems 

PM-ITTS 

Project  Manager  -  Instrumentation,  Targets,  and  Threat 
Simulators 

RGM 

Runtime  Global  Memory 

RS-232 

Recommended  Standard  232  (Serial  Interface,  IEEE) 

RSM 

Replicated  Shared  Memory 

RTI 

Runtime  Infrastructure 

SDD 

Software  Design  Description 

SRC 

Scientific  Research  Corporation 

SRS 

Software  Requirements  Specification 

STRICOM 

Simulations,  Training,  and  Instrumentation  Command 

SUM 

System/Software  Users  Manual 

TCP/IP 

Transmission  Control  Protocol/Internet  Protocol 

TRADOC 

Training  and  Doctrine  Command 

TBD 

To  Be  Determined 

TMS 

Threat  Minefield  System 

TDO 

Technical  Direction  Order 

TSMO 

U.S.  Army  Threat  Systems  Management  Office 

UDP 

User  Datagram  Protocol 

USB 

Universal  Serial  Bus 

UTM 

Universal  Transverse  Mercator 

UXO 

Unexploded  Ordnance 

VMM 

Virtual  Mine  Model 

1-124 


Appendices 


Appendix  A  Arc  Second  Laser-based  Position  Measurement  System 

A.  1  Concept  of  Operation 

The  Arc  Second,  Inc.  Constellation  3DI  Measurement  System  operates  by  triangulating 
the  position  of  a  photo  detector  relative  to  two  or  more  laser  transmitters.  Optimum  system 
performance  is  obtained  by  positioning  the  transmitters  to  ensure  good  triangulation  geometry  to 
the  desired  measurement  point.  For  the  TMS  system,  four  transmitters  should  be  positioned  so 
that  at  least  two  transmitters  are  visible  to  all  the  photo  detectors  at  all  times,  regardless  of 
operator  orientation.  The  recommended  optimal  work-area  is  the  square  configuration  as  shown 
in  Figure  A-l .  A  rectangular  or  trapezoidal  configuration  will  also  work  well. 

A.2  Setting  Up  The  Transmitters 

The  nominal  working  range  is  defined  as  the  longest  diagonal  distance  of  the  working 
zone.  The  nominal  working  range  should  be  approximately  the  maximum  range  of  the 
transmitters  to  the  photo  detectors.  The  Arc  Second  product  is  specified  to  work  out  to  35 
meters.  The  nominal  working  range  can  exceed  the  maximum  TX  to  RX  range  as  inferred  in 
Figure  A-3.  In  practice,  the  typical  maximum  range  is  a  function  of  several  factors  and  has  been 
observed  in  practice  out  to  50  meters.  Try  to  position  the  transmitters  so  that  the  measurement 
area  is  completely  enclosed  by  the  nominal  working  zone. 


v 


5 


Figure  A-l.  Work  Zone  Geometry 

To  ensure  good  geometry,  if  possible,  position  the  transmitters  so  there  is  an 
approximately  90°  convergence  angle  at  the  center  of  the  measurement  area  and  ensure  that  all 
desired  coverage  areas  lie  within  the  work  zone.  The  transmitters  are  numbered,  and  should  be 
placed  around  the  work  zone  in  a  counter  clockwise  numbered  fashion,  as  shown  in  Figure  A-l. 
This  will  provide  proper  orientation  of  the  measurement  coordinate  system,  following  the  right- 
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hand  rule,  with  the  positive  x-  axis  directed  from  Tx  1  to  Tx  2  and  the  positive  z-axis  90°  vertical 
from  the  x-y  plane.  Because  Tx  1  is  auto-leveling,  the  x-y  plane  is  always  perpendicular  to 
gravitational  pull.  Therefore,  elevation  differences  in  transmitters  do  not  affect  the  “tilt”  of  the 
x-y  plane  or  direction  of  the  z-axis. 

If  the  work  zone  is  interior  to  the  lines  connecting  the  transmitter  locations,  or  more 
basically,  if  the  transmitters  are  set  up  beyond  the  perimeter  of  the  desired  measurement  area, 
with  the  furthest  distance  between  any  two  transmitters  being  the  maximum  range  of  the 
transmitters,  it  is  most  likely  that  at  least  two  transmitters  will  be  visible  to  the  photo  detectors  on 
the  operator  at  all  times,  regardless  of  the  orientation  of  the  operator.  Measurement  accuracy 
improves  if  there  are  more  transmitters  visible  to  the  photo  detectors.  However,  as  long  as  two 
transmitters  are  visible,  acceptable  position  measurement  accuracy  is  achieved. 

The  transmitter  handle  orientation  should  be  set  toward  the  middle  of  the  working  zone  as 
illustrated  in  Figure  A-2. 


Toward 
center  of 
work  zone 


Figure  A-2.  Transmitter  Handle  Orientation 

If  the  transmitters  are  set  up  in  a  four-sided  configuration,  with  the  transmitters  oriented  as 
described  in  Figure  A-3,  the  areas  that  will  be  covered  by  at  least  two  of  the  transmitters  (i.e. 
where  an  unobstructed  receiver  will  be  within  range  of  reception),  will  be  similar  to  Figure  A-3. 
Therefore,  the  actual  total  coverage  area  extends  far  beyond  the  prescribed  work  zone.  However, 
if  an  operator  of  a  handheld  mine  detector  is  beyond  the  work  zone  with  his  back  to  the  work 
zone,  it  is  likely  there  will  be  frequent  position  measurement  dropouts.  Suitability  of  the  total 
coverage  area  beyond  the  work  zone  must  be  detennined  on  a  case-by-case  basis. 
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Figure  A-3.  Total  Coverage  Area  Geometry 

Laser  fan  beams  that  are  emitted  by  the  transmitters  cover  an  elevation  extent  of  +/-  45 
degrees.  Therefore,  there  is  an  area  just  under  each  transmitter  that  is  not  illuminated. 
Moreover,  if  a  photo  detector  is  too  close  to  a  transmitter,  the  photo  detector  is  saturated.  SRC 
recommends  a  minimum  separation  between  a  transmitter  and  photo  detector  of  20  feet.  It  does 
not  harm  the  photo  detector  or  electronics  to  be  within  the  minimum  suggested  separation.  The 
receiver  electronics  may  not  “recognize”  the  transmitter,  however. 

Each  transmitter  mounts  to  a  tripod  or  other  structure  using  a  5/8”  by  1 1  threads  per  inch- 
threaded  recess  in  its  base.  Secure  attachment  to  a  stable  base  is  important  to  prevent  transmitter 
movement  during  system  operation.  There  is  a  small  leveling  window  in  the  base  of  each 
transmitter  in  the  interior  of  the  handle.  This  can  be  used  to  level  each  transmitter.  Transmitter 
#1  has  a  self-leveling  mechanism,  which  will  level  the  transmitter  to  the  level  of  precision 
necessary  for  the  system  to  operate  properly.  The  remaining  three  transmitters  need  to  be 
relatively  level,  but  not  necessarily  perfectly  level  to  operate  properly.  A  transmitter  location 
will  be  unknown  if  it  is  moved  following  calibration.  Therefore  a  new  calibration  must  be 
performed  anytime  that  a  transmitter  is  displaced  from  its  original  pre-calibration  position.  It  is 
not  recommended  to  move  a  transmitter  while  it  is  spinning. 
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Transmitter  should 
be  level  and  stable 
for  best  system  perf oimance 


Figure  A-4.  Laser  Transmitter  Mount 
Table  A-l.  Constellation  3DI  Transmitter  Operation 


Transmitter  Item 

Item  Description 

Ready  LED 

Flashes  to  show  the  transmitter’s  operational  status  (Item  #1  in 
Figure  A- 5) 

Red  -  indicates  the  transmitter  is  on  but  not  yet  ready 

Treen  -  indicates  that  the  transmitter  is  ready  for  use  (i.e.  - 
spinning  at  the  prescribed  rate) 

Power  Button 

Turns  the  transmitter  on  and  off  (Item  #2  on  Figure  A-5) 

Auto  Leveling 

Push  items  3a  &  3c  (in  Figure  A-5)  at  the  same  time  and  hold 
them  3-5  seconds  until  an  LED  (3b)  is  blinking  (this  is  neither 
necessary  nor  recommended  for  normal  operation) 

Battery  Status  LED 

Flashes  to  show  the  approximate  charge  of  the  batteries  (Item  #4 
on  Figure  A-5) 

Green  -  indicates  that  the  batteries  are  ready 
-  indicates  that  the  batteries  are  low 

Red  -  indicates  that  batteries  are  too  low  to  operate  the 
transmitters 

Service  LED 

Flashes  red  when  the  transmitter  needs  servicing  (Item  #5  on 
Figure  A-5) 

Transmitter  batteries  do  not  need  to  be  removed  for  recharging.  The  recharger  can  be 
plugged  into  the  battery  tray  while  it  is  installed  in  the  transmitter.  However,  transmitter 
operation  is  disabled  during  charging.  The  transmitter  batteries  should  be  almost  or  fully 
discharged  before  re-charging.  The  re-charge  time  is  about  12  hours. 
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3a.  3b  3c,  4. 


■  ■ 


Figure  A-5.  Laser  Transmitter  Controls  and  Displays 
A.3  Calibrating  The  System 
A.3.1  Overview 

The  Constellation  3DI  Measurement  System  must  be  calibrated  before  any  measurements 
are  taken.  Calibration  is  the  process  of  determining  the  exact  location  and  orientation  of  each 
transmitter  on  the  measurement  site.  This  is  accomplished  by  using  calibration  software  and 
strategic  measurement  points  to  determine  the  position  and  orientation  of  each  of  the 
transmitters.  The  calibration  process  also  establishes  the  default  coordinate  reference  frame. 
After  calibration,  the  reference  frame  remains  intact  unless  the  transmitters  are  moved.  Only  a 
single  photo  detector  is  used  during  the  calibration  process. 

Once  the  transmitters  are  set  up,  turned  on  and  verified  through  3DiWorkbench  to  be 
visible  by  the  connected  photo  detector,  the  system  is  ready  for  calibration.  It  is  recommended 
that  during  the  calibration  process  the  3DiWorkbench  window  displaying  the  status  of  the  photo 
detector  remain  open  for  viewing.  All  transmitters  must  be  visible  to  the  photo  detector  used 
during  calibration. 

During  each  calibration  operation,  the  user  designates  various  points  within  the  coverage 
area  to  be  sampled  (measured)  by  the  Arc  Second  3DiWorkbench  calibration  software.  The 
calibration  points  should  be  distributed  throughout  the  work  zone  volume.  Usually  a  point  is 
taken  between  each  of  the  transmitters  on  the  edge  of  the  work  zone  and  then  2  to  4  more  within 
the  work  zone  in  addition  to  two  more  points  for  the  scalebar  for  a  total  of  8-10  points  measured 
(see  Figure  A-7).  It  is  important  to  hold  the  optical  receiver  very  still  during  calibration  (it  is 
recommended  that  during  calibration  the  photo  detector  be  placed  on  a  stationary  object).  It  is 
also  important  that  the  receiver  be  in  the  vertical  orientation. 

3DiWorkbench  monitors  the  standard  deviation  of  the  measured  angles  from  the 
transmitters.  Each  time  that  a  calibration  point  is  collected,  a  result  dialog  will  display  the 
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standard  deviation  for  each  transmitter  in  micro  radians.  These  standard  deviations  need  to  be 
monitored  carefully.  The  nominal  value  for  a  successful  calibration  will  be  less  than  100 
microradians.  If  the  displayed  standard  deviation  for  any  transmitter  is  in  excess  of  150 
microradians,  click  the  Stop  and  Reset  buttons  and  recollect  the  angles  for  that  point. 

Scale  is  introduced  into  the  measurements  by  using  a  scalebar.  A  scalebar  is  entered  by 
specifying  an  exact  measurement  of  the  distance  between  two  corresponding  calibration  points  in 
meters.  The  greater  the  distance  between  the  2  points  used  in  conjunction  with  the  scalebar,  the 
more  accurate  the  position  measurements  will  be. 

It  is  recommended  that  the  photo  detector  used  to  collect  calibration  points  not  be  at  the 
same  vertical  height  as  the  transmitters  (see  Figure  A-6).  In  other  words,  try  to  avoid  the  plane 
of  the  transmitters  during  calibration.  This  will  provide  a  more  mathematically  precise 
calibration. 


Figure  A-6.  Transmitter  Plane 


Figure  A-7.  Example  Calibration  Point  Layout 
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Sample  Location  1  o(6 

Move  to  the  sample  location,  and  click  Begin  to  begin  collecting  data. 
Stop  or  Next  to  move  to  the  next  sample  location  point. 
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Figure  A-8.  3DiWorkbench  during  Calibration 


A. 3. 2  Calibration  Steps 


Note:  See  3DIWorkBenchUser  Guide  for  information  on  configuring  PCEs,  Input  Devices  and 
Position  Servers. 


1 .  Position  the  transmitters  around  the  work  area  and  turn  them  on. 

2.  Attach  a  single  detector  to  the  left  foot  port  on  the  FIU  (Note:  This  is  the  only  device  that 
must  be  attached  during  the  calibration  process).  Power  on  the  FIU. 

3.  Power  on  the  Evaluator  Workstation.  Launch  PCAnywhere  and  establish  a  connection  to  the 
FIU  (Note:  Subsequent  instructions  regarding  software  operations  on  the  FIU  imply  using  the 
PCAnywhere  connection.) 

4.  On  the  FIU,  launch  3DiWorkbench. 

5.  Within  3DiWorkbench,  from  the  File  menu,  select  Open.  From  the  resulting  file  selection 
dialog,  select  the  file  \TMS_Home\TMS_Bin\FIU.3Di. 
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6.  From  the  3Di  Tree  view  (left  pane)  display,  click  on  the  Input  Device  branch.  In  the 

resulting  Input  Devices  Property  Section  (right  upper  pane),  under  the  Properties  tab,  click 
the  Start  All  button.  Non-zero  position  values  should  appear  in  the  AsiServerSinglePoint 
dialog  entitled  “0”  (See  Figure  A-9). 
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Figure  A-9.  Example  of  AsiServerSinglePoint  Dialog  Entitled  “0” 

7.  Position  the  detector  at  a  point  midway  between  transmitters  1  and  2,  slightly  inset  into  the 
work  area.  From  3DiWorkbench,  using  the  “0”  dialog,  ensure  that  all  transmitters  are  visible 
as  indicated  by  the  “B”  (for  blocked)  field:  0  should  appear  if  all  transmitters  are  visible, 
otherwise  Tx{n}  (where  n  =  transmitter  #)  will  appear  if  Tx{n}  is  not  visible. 

8.  Under  3DiWorkbench,  from  the  Setup  menu,  select  Perform  Setup. . . .  This  should  display 
the  3DiWorkbench  Setup  Wizard  (see  Figure  A- 10).  Click  Next. 
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Figure  A- 10.  3DiWorkbench  Setup  Wizard 

9.  On  the  Step  1 :  Settings  for  the  Setup  page,  enter  the  Number  of  Sample  Locations  as  6  and 
the  Number  of  Scale  Bars  as  1  (see  Figure  A-l  1).  Click  Next. 
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Step  1 :  Settings  for  the  setij) 

Specify  the  particular  settings  relevant  to  your  environment  and  how  you 
would  like  to  perform  the  setup. 


Number  of  Sample  Locations: 
Number  of  Scale  Bars: 


"3 
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Advanced  Settings... 


<  Back 
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Cancel 
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f  Start  Q  C:\Documents  and  ...  &  3DIWorkbench  -  FI... 


Figure  A-l  1 .  Step  1 :  Settings  for  the  Setup  Page 

10.  On  the  Sample  Location  1  of  6  page,  verify  that  Input  Device  is  II  and  Detector  is  Dl.  Click 
the  Begin  button.  Allow  the  collection  of  at  least  100  samples.  While  collecting,  verily  that 
the  displayed  value  of  StdDev2  for  each  transmitter  stays  above  zero  and  below  100 
microradians  (.0001  radians  as  displayed).  Click  the  Stop  button  after  collecting  sufficient 
samples.  If  the  value  of  StdDev2  for  any  transmitter  exceeds  100  microradians  or  any 
transmitter  is  indicated  as  not  visible  in  the  “0”  dialog  during  data  collection,  click  the  Reset 
button  and  recollect  the  samples  for  that  point.  When  the  sufficient  samples  for  Point  1  have 
been  collected,  click  the  Next  button  (see  Figure  A- 12). 
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Sample  Location  1  of  6 

Move  to  the  sample  location,  and  click  Begin  to  begin  collecting  data.  Click 
Stop  or  Next  to  move  to  the  next  sample  location  point. 
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Figure  A-12.  Sample  Location  1  of  6  Page 


11.  Repeat  step  10  for  Sample  locations  2-4,  moving  the  detector  between  each  next  successive 
set  of  2  transmitters  (e.g.  Point  2  between  transmitters  2  and  3,  Point  3  between  transmitters  3 
and  4,  Point  4  between  transmitters  4  and  1). 

12.  See  step  11. 

13.  See  step  11. 

14.  For  Sample  Locations  5  and  6,  place  the  detector  at  any  2  arbitrary  points  within  the  work 
area  and  collect  sample  data  as  described  in  step  10. 

15.  See  step  14. 

16.  After  collecting  Sample  Location  6,  the  Scale  Bar  1  Distance  page  should  appear.  Mark  2 
points  within  the  work  area  and  accurately  measure  the  distance  between  the  points.  Enter 
that  value  in  the  Scale  Bar  Length  field  (Note:  The  distance  should  be  at  least  one  meter  but 
need  not  be  greater  than  three  meters  (see  Figure  A- 13).  Click  Next. 


1-135 


FIU3  -  Symantec  pcAnywhere  Remote 


File  Edit  Task  Actions  Help 


^  File  Transfer 
Q  Command  Queue 
a  Show  Chat 
x  End  Session 


Remote  Control 

9  —  „ 

Screen  Scaling 
^  View/Edit  Online  Options 
1^  T  ransfer  Clipboard 

S tart/S top  Session  Recording 
Save  Current  Screen 
3*  Send  Ctrl+Alt+Del 
@  Restart  Host  Computer 


5 


Details 

$3  FIU3 

169.254.39.10 

TCP/IP 

$  Encryption  Level:  None 
Windows  XP 
Connected 

TimeofSession:00:41:33 

a  Send:  163  KB 

Receive:  2892  KB 


* 


Remote  Control 


& 


IS 


Setup  Transmitters 


E 


Scale  Bari  Distance 

P]  Enter  the  known  length  of  the  scale  bar  you  will  measure  as  part  of  this 

setup. 


Con 


Scale  Bar 


Length:  li.  22 


] 


f  Start  Q  C:\Documents  and  . 


Z  1-1.05223955 

M:]0.00809221 

?:  In 


Z  |0.00000000 
M:  la 00000000 


A  3DIWorkbench  -  FI... 


Figure  A-13.  Scale  Bar  1  Distance  Page 

17.  Place  the  detector  on  one  of  the  measured  points.  On  the  Sample  First  Point  of  the  Scale  Bar 
1  of  1  page,  click  the  Begin  button,  following  the  procedures  in  step  10  (see  Figure  A-14). 
Click  Next. 
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Figure  A- 14.  Sample  First  Point  of  the  Scale  Bar  1  of  1  Page 


18.  Place  the  detector  on  the  second  of  the  measured  points.  Repeat  step  17  on  the  Sample 
Second  Point  of  Scale  Bar  1  of  1  page.  Once  the  sufficient  samples  are  collected,  click  Next. 

19.  On  the  Calculate  Setup  Bundle  page,  click  the  Calculate  button.  If  the  collected  data  is 
sufficient,  a  dialog  stating  that  the  Bundle  Calculation  succeeded  will  be  displayed  (see 
Figure  A- 15).  Otherwise,  the  dialog  will  indicate  that  the  bundle  calculation  failed.  In  this 
case  the  setup  procedures  must  be  repeated.  Click  Ok  to  close  the  results  dialog.  Click  the 
Accept  Bundle  button.  A  dialog  will  then  appear  with  a  prompt  to  send  the  new  setup  to  all 
connected  devices.  Click  the  Yes  button  (see  Figure  A- 16).  Click  Next. 
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Figure  A- 15.  Calculate  Setup  Bundle  Page 
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Figure  A- 16.  Prompt  to  Send  Bundle  Data  to  All  Connected  Devices 
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20.  The  final  Setup  Wizard  page  will  appear  (see  Figure  A- 17).  Click  Finish. 
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Figure  A- 17.  Setup  Wizard  Final  Page 

2 1 .  In  3DiWorkbench,  in  the  3Di  Tree  view  (left  pane)  display,  the  Configurations  branch  should 
include  a  Setup  entry  showing  the  current  data  and  time. 

22.  Within  3DiWorkbench,  from  the  File  menu,  select  Save. 

This  completes  the  calibration  process.  The  corresponding  work  area  will  remain  “calibrated”  as 
long  as  the  transmitters  are  not  moved.  3DiWorkbench  can  be  exited  at  this  time  if  necessary 
and  the  FIU  shutdown. 
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Appendix  B  Demonstration  Unit  Operations 


B.l  Scope 
B .  1 . 1  Identification 

Scientific  Research  Corporation  (SRC)  under  Contract  DAAH01-00-C-A107  Technical 
direction  Order  (TDO)  0029  of  the  US  Army  Threat  Systems  Management  Office  (TSMO) 
program,  was  contracted  by  TSMO  to  develop  the  Threat  Minefield  System  (TMS)  Demo  Unit 
(DU)  in  parallel  with  the  TMS  Phase  III  effort  (TDO  0028  of  the  same  contract).  This  Appendix 
to  the  TMS  System  Users  Manual  (SUM)  in  conjunction  with  the  TMS  SUM  describes  in 
sufficient  detail  the  provided  interfaces  and  procedures  necessary  for  operation  of  the  TMS  DU. 
This  Appendix  pertains  to  all  computer  software  configuration  items  (CSCI)  mentioned  in 
section  1.1  of  the  TMS  SUM  as  well  as  the  following  TMS  DU  specific  computer  software 
configuration  item  applications: 

•  7000  Utility  version  4.2.0  light  controller  configuration  application 

B.  1 .2  System  Overview 

The  TMS  DU  is  an  augmentation  of  the  TMS  test  and  training  capabilities.  The 
augmentation  facilitates  demonstration  of  the  TMS  capabilities  to  audiences  either  indoors  or 
outdoors  using  an  elevated  test  lane  with  audible  and  visual  feedback  for  exercise  events.  The 
TMS  DU  also  facilitates  full  operation  of  the  single-operator  mobile  version  of  the  TMS 
whereby  exercises  and  training  can  be  perfonned.  This  includes  the  application  of  virtual  mines. 

The  TMS  DU  represents  work  performed  in  all  previous  and  concurrent  TMS 
development  phases.  As  with  the  primary  TMS  system,  software  configuration  management 
(CM)  is  perfonned  for  the  US  Government  by  SRC  using  the  Microsoft  (MS)  CM  tool  Visual 
SourceSafe. 

The  primary  hardware  components  of  the  TMS  DU  consist  of: 


•  DataComms  /  Evaluator  Workstation  Laptop  Computer 

•  Countermine  Test  Management  System  (CTMS)  Laptop  Computer 

•  Elevated  and  Desktop  Display  Equipment 

•  Operator  Instrumentation 

•  Position,  Locating  and  Tracking  Equipment 

•  Elevated  Test  Lane  Equipment. 

All  of  the  above  referenced  hardware  is  common  to  the  single  evaluator  TMS  hardware  suite 
with  the  exception  of  the  display  and  elevated  test  lane  equipment. 

B .  1 . 3  Appendix  Overview 
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This  appendix  applies  only  to  the  TMS  DU  audience  demonstration  functions  of  the  TMS 
DU  product.  The  single  evaluator  functions  and  procedures  are  documented  in  the  TMS  SUM 
document  main  body.  This  appendix  has  two  primary  sections.  The  first  section  entitled 
“Hardware  Features  for  Demonstration  Unit  Operation”  provides  instructions  for  the  set-up  and 
use  of  the  hardware  specific  to  the  TMS  DU  operation.  The  second  section  entitled  “Software 
Features  for  Demonstration  Unit  Operation”  provides  instructions  for  the  initialization  and  use  of 
the  software  specific  to  the  TMS  DU  operation. 


B.2  Hardware  Features  for  Demonstration  Unit  Operation 

This  section  contains  descriptions  of  and  instructions  on  the  use  of  the  features  of  the 
TMS  DU  hardware  that  specifically  support  operations  of  the  Demonstration  Unit.  All  other 
hardware  operation  instructions  common  to  normal  operation  of  the  TMS  system  are  provided  in 
the  SUM  main  body. 

B.2. 1  Elevated  Test  Lane  Equipment 

The  elevated  test  lane  is  composed  of  a  non-metallic  structure  to  support  an  operator  and 
his  gear  for  TMS  demonstration  purposes.  The  test  lane  supports  targets  that  can  be  concealed 
under  translucent  white  acrylic  sheets  just  below  the  top  grating.  The  lane  is  configurable  in  that 
from  1  to  10  sections  of  the  lane  can  be  assembled  to  adjust  the  size  of  the  demonstration 
platform  dependant  upon  the  space  available.  The  elevated  test  lane  drawings  are  contained 
within  the  drawing  package  provided  with  the  TMS  Demo  Unit  system  and  will  not  be 
duplicated  here.  However,  the  assembly  and  specific  parts  are  readily  identifiable  in  the 
Hardware  Drawing  Tree.  A  description  of  their  use  is  provided  here. 

B .2 . 1 . 1  Elevated  T est  Lane  Sections 

B.2. 1.1.1  Geometry 

Each  elevated  test  lane  section  is  three  feet  wide  and  four  feet  long.  When  two  sections 
of  three  feet  width  are  joined,  the  intended  width  of  six  feet  is  realized.  It  is  not  recommended 
that  sections  be  used  individually  to  form  a  three-foot  wide  width  because  of  operations  and 
safety  issues.  The  widths  of  three  feet  were  provided  for  ease  of  handling  and  shipping  purposes. 
When  two  sections  are  joined  for  a  width  of  six  feet,  lengths  in  increments  from  four  feet  up  to 
twenty  feet  can  be  constructed. 

Each  elevated  test  lane  section  is  composed  of  a  frame,  a  clear  acrylic  sheet  with  stand¬ 
offs  for  target  support,  a  translucent  white  acrylic  sheet  for  covering  the  targets  (if  desired),  black 
grating,  exterior  translucent  white  side  panels  with  knobs  and  interconnecting  pins.  The  top 
acrylic  sheet  covering  the  targets  is  optional  depending  on  the  intent  of  demonstration.  If  it  is 
desired  that  the  target  locations  be  seen,  it  can  be  left  off.  An  isometric  of  the  single  cube 
assembly  as  is  depicted  in  drawing  number  6HTS2-00003  is  provided  below  in  Figure  B-l.  It  is 
shown  without  side  panels. 
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Figure  B-l.  Elevated  Test  Lane  Cube  Assembly  without  Side  Panels 


B.2. 1.1.2  Target  Placement 

The  targets  (mines,  simulants  or  other  metallic  clutter)  may  be  placed  on  the  clear  acrylic 
sheet  before  the  other  white  acrylic  sheet  or  grating  is  put  in  place.  Target  depth  is  controlled  by 
placing  non-metallic  support  under  the  emplaced  target  to  raise  it  to  the  intended  depth  below  the 
upper  surface  of  the  grating.  The  grating  is  two  inches  thick  to  support  an  operator  and  his  gear 
safely  and  the  top  white  acrylic  sheet  is  3/16  inch  thick.  Therefore,  any  target  touching  the 
bottom  of  the  white  acrylic  top  sheet  is  2  and  3/16  inches  “deep”  which  is  relatively  deep  for  an 
anti-personnel  mine  or  low-metallic  simulant.  Alternatively,  simulant  inserts  will  fit  within  the 
2-inch  by  2-inch  grating  and  will  rest  on  top  of  the  top  white  acrylic  sheet.  This  will  enhance  the 
target  response  but  will  expose  the  target  location.  Alternatively,  coins  or  other  metal  may  be 
placed  with  targets  under  the  top  white  acrylic  sheet  to  “enhance”  the  mine  detector  response  to 
help  compensate  for  the  depth. 

Suggestion:  Before  placing  the  top  white  acrylic  or  grating,  perfonn  the  necessary  survey 
to  develop  the  ground  truth  file  (see  Section  B.3.2.3).  The  laser  detector  used  can  then  be 
positioned  exactly  where  the  real  or  virtual  mine  is  to  be  placed  thereby  negating  the  need  to 
measure  any  depth  offsets. 


B  .2 . 1 .2  Elevated  T est  Lane  Flandrail 

The  elevated  test  lane  handrail  should  be  used  for  safety  purposes  after  the  position, 
locating  and  tracking  system  is  calibrated  and  all  other  pre-demonstration  work  is  complete.  The 
handrail  serves  to  help  link  the  exterior  of  adjoining  sections  together.  The  handrail  is  not 
intended  to  meet  any  workplace  handrail  or  foot  rail  requirements  and  will  not  support  the 
weight  of  a  human.  The  handrail  is  to  serve  as  a  reminder  where  the  edge  of  the  elevated  lane  is 
if  it  is  backed  into  or  leaned  into.  When  operating  a  detector,  it  is  common  to  lose  cognizance  of 
the  location  of  the  edge  of  the  platform,  as  detector  operation  is  mentally  absorbing.  If  it  is 
bumped  into,  it  will  serve  as  a  reminder  to  the  operator  where  the  edge  is. 
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At  each  six-foot  wide  end  of  the  test  lane,  the  handrail  slides  open  and  closed  to  fonn  a 
gate  of  sorts  and  allow  access  to  the  lane.  Always  remember  to  close  the  “gate”  as  the  most 
important  use  of  the  railing  is  to  deter  operators  from  backing  up  too  far.  Again,  the  handrail 
will  not  support  the  weight  of  an  operator  but  will  provide  a  reminder  of  where  the  edge  of  the 
test  lane  is. 

The  handrail  may  also  undesirably  block  the  position,  locating  and  tracking  laser 
transmitter  emissions  at  certain  points  on  the  test  lane.  Experimenting  with  transmitter 
positioning  to  avoid  the  “shadowing”  that  may  occur  will  result  is  more  “intelligent”  positioning 
of  the  transmitters  relative  to  the  test  lane  and  hand  railing.  Although  there  are  four  transmitters 
simultaneously  used,  position  detectors  only  need  to  have  two  transmitters  visible  at  any  one 
location.  Therefore,  blocking  during  operation  is  not  a  serious  problem  but  can  impact 
positioning  signal  processing  momentarily  resulting  in  a  very  short  but  noticeable  positioning 
“transient”.  Since  positioning  system  calibration  requires  line  of  sight  between  the  calibration 
detector  and  all  four  transmitters  simultaneously,  the  handrail  should  not  be  put  up  before 
calibration  is  complete. 

B.2. 1 .3  Light  Trees  and  Controllers 

A  light  tree  and  controller  is  used  for  each  elevated  test  lane  segment.  Power  to  the  lights 
are  provided  through  the  controller  which  is  controlled  by  software  running  on  the  DataComms  / 
Evaluator  workstation  laptop.  Power  and  control  lines  are  provided  through  a  “daisy-chain”  type 
of  network  from  one  to  the  other.  The  light  trees,  controllers  and  associated  control  and  power 
cables  should  be  positioned  BEFORE  any  of  the  acrylic  sheets  or  grating  is  put  in  place. 

Each  light  tree  should  be  placed  on  the  floor  and  either  positioned  under  the  center  of  the 
test  lane  segment  or  be  placed  directly  under  any  target  (if  so  desired).  If  two  or  more  targets  are 
placed  within  a  single  section,  the  light  tree  will  probably  be  better  if  placed  in  the  middle  of  the 
test  lane  section.  The  controller  for  each  can  be  placed  wherever  it  is  convenient  but  should  be 
placed  in  the  same  test  lane  section.  Since  the  light  trees  are  placed  before  the  targets  and 
supporting  acrylic  sheets  are  positioned,  fine  adjustment  of  the  light  tree  positions  may  be 
necessary  via  the  side  of  the  sections. 

Each  light  tree  controller  is  addressed  and  correspondingly  numbered  for  software 
control.  Controller  1  MUST  be  positioned  in  the  test  lane  section  that  is  closest  to  Transmitter  1 
of  the  positioning  hardware  under  the  LEFT  hand  three-foot  wide  section.  Controller  2  will  be 
placed  under  the  RIGHT  hand  three-foot  wide  section  adjacent  to  Controller  1.  The  controller 
placement  will  then  follow  the  same  pattern  as  indicated  in  Figure  B-2  below.  In  the  event  that 
fewer  than  all  ten  test  lane  sections  are  used,  the  higher  numbered  controllers  will  be  left  out.  It 
is  not  necessary  that  the  controller  data  cable  and  power  cables  be  daisy-chained  in  numerical 
order.  In  fact,  the  recommended  daisy-chain  order  is  from  1  to  3  to  5  and  so  on  around  the  test 
lane  sections  clockwise  ending  with  6  to  4  to  2. 
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Figure  B-2.  Light  Tree  Controller  Numbering  and  Placement 
B.2.1.4  Audio  Equipment 

The  audio  hardware  provided  includes  a  subwoofer,  four  surround- sound  speakers  (2- 
firont  and  2-rear),  a  fifth  speaker  for  the  center  and  a  controller.  The  subwoofer  chassis  contains 
all  the  audio  electronics  for  amplifying  the  sound  and  distributing  the  signals  to  the  five  ancillary 
speakers.  The  audio  gear  is  driven  with  an  input  from  the  speaker  jack  of  the  DataComms  / 
Evaluator  Workstation  laptop.  The  laptop  speaker  port  is  connected  to  the  controller  input  where 
the  audio  is  processed.  The  controller  is  adjusted  for  the  “Stereo  x  2”  selection.  The  .WAV  file 
played  during  an  exercise  is  that  of  an  explosion  if  a  mine  is  virtually  “detonated”  by  an  operator 
stepping  on  it.  The  controller  output  is  connected  to  subwoofer  for  audio  amplification. 

Each  of  the  four  surround  sound  speakers  is  provided  with  mounts  that  can  be  mounted 
on  the  top  corners  of  the  handrail.  Fifty  feet  of  speaker  wire  with  banana  plug  connections  are 
provided  for  each  speaker  to  allow  cable  routing  as  necessary  for  the  demonstration  and  available 
space.  The  speaker  mounts  do  not  have  to  be  used  but  serve  as  a  convenient  way  to  surround  the 
operator.  The  center  and  subwoofer  speakers  should  be  collocated  in  close  proximity  to  the 
evaluator  workstation  laptop  as  the  controller  is  connected  to  the  laptop.  The  controller  has  knob 
and  pushbutton  controls  for  audio  adjustment  such  as  muting,  volume,  fade  and  balance. 

B.2.2  Elevated  Display  Equipment 

The  elevated  display  equipment  is  composed  of  two  42-inch  plasma  monitors,  two  19- 
inch  plasma  tabletop  monitors,  associated  stands,  video  splitters  and  cables.  The  monitor  stands 
are  designed  such  that  the  bases  can  have  wheels  mounted  to  the  bottom  so  the  displays  can  be 
easily  moved.  The  display  bases  can  be  tied  together  for  additional  stability  and  ease  of  use. 
The  display  mounts  are  adjustable  so  the  display  can  be  easily  pivoted  around  (no  tools  required) 
and  can  be  tilted  up  or  down  (with  loosening  and  tightening  mounting  hardware).  The  display 
and  mounting  drawings  are  contained  within  the  drawing  package  provided  with  the  TMS  Demo 
Unit  system  and  will  not  be  duplicated  here.  However,  the  assembly  and  specific  parts  are 
readily  identifiable  in  the  Hardware  Drawing  Tree.  A  single  elevated  display  isometric  as  is 
provided  in  drawing  number  4HTS2-00007  is  provided  below  in  Figure  B-3.  The  wheels  are 
shown  mounted  on  the  same  side  of  the  base  as  the  display.  It  is  possible  and  may  be  desirable 
to  mount  them  on  the  opposite  side  to  allow  the  display  to  roll. 
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Figure  B-3.  Single  Elevated  Display  Assembly  Isometric 


The  Evaluator  Workstation  laptop  display  output  port  and  CTMS  laptop  display  output 
port  will  each  be  connected  to  the  input  of  a  provided  video  splitter.  The  video  splitter  outputs 
can  be  connected  to  both  of  the  overhead  plasma  displays  as  well  as  to  one  each  of  the  tabletop 
displays  (when  all  the  displays  are  used).  The  video  displays  used  will  be  a  function  of  how 
much  room  is  available  and  the  presumed  audience  size.  The  displays  can  be  used  in  any 
combination  and  any  configuration. 

Each  of  the  42-inch  displays  supports  two  display  inputs.  Through  an  RS-232  control  the 
displays  can  be  controlled  to  select  which  video  port  to  display  or  simultaneously  view  both  with 
a  “picture  in  picture”  mode.  The  software  to  control  the  displays  is  installed  on  the  Evaluator 
Workstation.  The  additional  RS-232  ports  are  provided  through  USB  adapters  and  a  USB  bus 
extender. 

B.3  Software  Features  for  Demonstration  Unit  Operations 

This  section  contains  descriptions  of  and  instructions  on  the  use  of  the  features  of  the 
TMS  workstation  applications  that  specifically  support  operations  of  the  DU.  The  features 
unique  to  DU  operations  are  available  only  when  the  system  is  configured  in  “Demo  Mode.”  The 
DU  utilizes  the  TMS  evaluator  workstation  applications  configured  in  single  evaluator  mode. 
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B.3.1  TMS  DataComms 


The  TMS  DataComms  application  implements  the  data  communications  interface 
through  which  commands  are  sent  to  and  responses  are  received  from  the  DU  light  controllers. 
This  interface  is  facilitated  through  the  built-in  RS-232  serial  port  (COM1)  of  the  TMS  evaluator 
workstation.  When  appropriate  exercise  events  occur  as  detected  by  TMS  MISP,  TMS 
DataComms  sends  a  set  of  commands  indicating  the  desired  light  behavior  to  the  light  controller 
corresponding  to  the  event  position  on  the  elevated  lane  platform. 

B.3.1.1  Start  Up  for  Demo  Mode 

TMS  DataComms  is  initialized  for  Demo  Mode  at  start  up  by  the  use  of  a  command  line 
parameter:  /d.  TMS  DataComms  can  be  started  stand-alone  or,  if  it  is  not  currently  running,  by 
TMS  Master  when  it  is  started.  To  start  TMS  DataComms  stand-alone  in  Demo  Mode,  create  a 
Windows  shortcut  to  the  TMS  DataComms  application;  right-click  on  the  shortcut  and  select 
“Properties”;  at  the  end  of  the  “Target”  field  on  the  Shortcut  tab,  add  a  space  and  /d;  close  the 
Properties  dialog  by  selecting  OK.  This  step  must  be  performed  only  once,  as  long  as  the 
shortcut  is  never  deleted.  TMS  DataComms  may  then  be  launched  by  double  clicking  the 
shortcut.  Demo  Mode  configuration  is  confirmed  by  “Demo  Mode”  in  the  title  bar  of  the  TMS 
DataComms  main  window.  It  is  also  confirmed  by  opening  the  About  box  for  TMS  DataComms 
(as  well  as  all  other  TMS  evaluator  workstation  applications  running  in  Demo  Mode).  The  About 
box  should  appear  as  shown  in  Figure  B-4,  where  Demo  Mode  is  indicated  with  the  version 
number. 


Figure  B-4.  TMS  DataComms  About  box  showing  Demo  Mode  operation. 
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B.3. 1 .2  Light  Controller  Operation  Test 


Proper  configuration  and  operation  of  the  light  controllers  for  the  elevated  platform  can 
be  confirmed  using  TMS  DataComms  by  selecting  the  “Test  Light  Controllers...”  item  of  the 
System  menu.  Upon  selection  of  this  item  (which  is  enabled  only  when  no  exercise  is  currently 
configured  on  the  workstation),  the  dialog  in  Figure  B-5  is  displayed. 


Figure  B-5.  TMS  DataComms  Demo  Mode  light  controller  test  dialog. 

The  user  input  items  on  this  dialog  allow  the  specification  of  the  number  of  light 
controllers  to  test  (which  should  correspond  to  the  number  of  elevated  lane  segments)  as  well  as 
which  lights  should  be  illuminated  by  each  controller.  Any  or  all  of  the  lights  may  be  selected.  In 
addition,  it  allows  the  optional  specification  of  whether  the  test  should  stop  on  any  failure  and 
whether  the  test  should  loop  continuously  until  stopped  by  the  user.  Clicking  on  the  Start  button 
initiates  the  test.  (After  the  test  is  started,  the  Start  button  becomes  the  Stop  button.)  The  test  can 
be  stopped  at  any  time  by  clicking  the  Stop  button  or  by  closing  the  dialog.  The  status  of  each 
command  sent  to  each  controller  is  displayed  in  the  Status  list  box.  Note:  a  command  will  fail  if 
a  light  controller  is  missing  or  not  properly  connected;  a  command  will  not  fail  if  the 
corresponding  light  bulb  is  not  powered,  missing  or  inoperable. 

Additional  light  controller  diagnostics  may  be  performed  using  the  7000  Utility 
application  provided  by  the  light  controller  manufacturer.  This  application  is  installed  on  the  DU 
evaluator  workstation.  It  provides  an  online  Flelp  feature  for  usage  instructions.  Note:  this 
application  is  also  used  to  program  the  “address”  of  each  controller;  the  address  corresponds  to 
the  ID  number  of  the  controller  indicated  on  its  outer  case.  Normal  DU  operations  will  not 
require  that  the  controller  addresses  be  re-programmed. 
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B.3. 1 .3  Instrumentation  Survey  Interface 

TMS  DataComms  implements  the  Instrumentation  Survey  (“Instr  Survey”)  data 
communications  interface  to  support  test  lane  ground  truth  file  creation  using  the  Arc  Second 
positioning  system  in  conjunction  with  the  TMS  FIU.  Ground  truth  files  used  in  Demo  Mode 
exercises  can  be  created  using  the  Survey  operation  of  TMS  Master  where  the  survey  entity  used 
is  an  FIU  connected  through  an  Instr  Survey  interface.  Prior  to  initiating  the  Survey  operation, 
create  the  interface  to  the  FIU  as  an  Instr  Survey  interface.  If  an  interface  of  another  type  to  the 
FIU  currently  exists,  that  interface  must  first  be  deleted.  The  details  of  creating  a  ground  truth 
file  for  a  Demo  Mode  exercise  are  provided  in  the  following  section  on  TMS  Master. 

B.3.2  TMS  Master 

The  TMS  Master  application  includes  Demo  Mode  capabilities  specific  to  executing  an 
exercise  on  the  elevated  platform. 

B.3.2. 1  Start  Up  for  Demo  Mode 

TMS  Master  is  initialized  for  Demo  Mode  at  start  up  by  the  use  of  a  command  line 
parameter:  /d.  To  start  TMS  Master  in  Demo  Mode,  create  a  Windows  shortcut  to  the  TMS 
Master  application;  right-click  on  the  shortcut  and  select  “Properties”;  at  the  end  of  the  “Target” 
field  item  on  the  Shortcut  tab,  add  /d  and  close  the  Properties  dialog  by  selecting  OK.  This  step 
must  be  perfonned  only  once,  as  long  as  the  shortcut  is  never  deleted.  TMS  Master  may  then  be 
launched  by  double  clicking  the  shortcut.  Demo  Mode  configuration  is  confirmed  by  “Demo 
Mode”  initially  in  the  title  bar  of  the  TMS  Master  main  window.  It  is  also  confirmed  by  opening 
the  About  box  for  TMS  Master  as  described  previously.  TMS  Master  will  launch  TMS 
DataComms  in  Demo  Mode  if  it  was  not  started  stand-alone. 

B.3.2.2  Elevated  Platform  Test  Lane  Dimensions 

The  dimensions  of  the  “test  lane”  represented  by  the  elevated  platform  must  be  specified 
prior  to  conducting  exercises.  After  it  is  launched,  TMS  Master  will  display  the  dialog  shown  in 
Figure  B-6. 


Figure  B-6.  Set  Demo  Lane  Dimensions  dialog 
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This  dialog  allows  the  specification  of  the  dimensions  of  the  elevated  platform  test  lane  in  terms 
of  segments.  The  default  dimensions  are  five  segments  long  by  two  segments  wide.  The 
dimensions  specified  at  this  time  will  be  in  effect  for  the  duration  of  the  TMS  execution  session. 

B.3.2.3  Elevated  Platform  Test  Lane  Survey 

The  ground  truth  data  used  in  a  Demo  Mode  exercise  conducted  on  the  elevated  platform 
is  collected  using  the  Survey  operation  of  TMS  Master.  Prior  to  initiating  the  Survey,  an 
Instrumentation  (“Instr”)  Survey  data  communications  interface  to  the  DU  FIU  must  be 
established  within  TMS  DataComms.  Refer  to  the  previous  section  on  TMS  DataComms  for 
more  information  on  Instr  Survey  interfaces.  Refer  to  section  5.3.2.2.4  of  the  System  Users 
Manual  on  TMS  Master  Survey  operations  for  general  infonnation  on  configuring  and 
conducting  survey  operations.  The  position  data  received  from  an  FIU  connected  through  an 
Instr  Survey  interface  used  for  ground  truth  object  measurements  corresponds  to  that  obtained 
through  the  Arc  Second  sensor  designated  as  “Left  Foot.” 

The  boundaries  of  the  test  lane  associated  with  the  elevated  platfonn  are  designated  using 
the  standard  lane  boundary  marker  types.  These  markers  may  coincide  with  the  platfonn  comers 
or  they  may  be  inset  from  the  corners.  In  the  latter  case,  the  comers  can  be  marked  using  a  pre¬ 
defined  set  of  Landmark  identifiers.  These  are,  specifically,  “DEMO  F  L”  (front  left), 

“DEMO  F  R”  (front  right),  “DEMO  B  L”  (back  left)  and  “DEMO  B  R”  (back  right).  If 
specified,  the  overall  platfonn  area  will  be  calculated  using  these  landmarks;  otherwise,  the 
platform  area  will  be  calculated  using  the  beginning-  and  end-of-lane  markers. 

Once  the  lane  boundary  locations  are  measured,  the  location  of  virtual  mines  may  be 
specified  manually  within  the  Survey  operation.  Double  clicking  on  the  Survey  grid  display  will 
open  a  Create  New  Object  dialog  with  the  northing  and  easting  values  filled  in  for  the  location 
clicked.  The  altitude  value  is  by  default  set  to  that  specified  for  the  survey  data  reference  point. 
The  survey  data  reference  point  may  be  set  from  measured  position  data  using  the  Message 
button  on  the  Survey  Data  modification  dialog.  The  altitude  value  for  the  reference  point 
provided  from  the  measured  data  must  then  have  subtracted  from  it  the  height  of  the  center  of  the 
Arc  Second  sensor  above  the  platform  grid.  Assuming  that  the  platform  is  relatively  level,  the 
altitude  value  for  a  manually  entered  virtual  mine  can  be  specified  as  the  default  altitude  value 
less  the  desired  “burial  depth.”  Note:  per  convention,  the  burial  depth  value  is  specified  in 
inches,  the  altitude  is  specified  in  meters. 

B.3.2.4  Metal  Detector  Background  Calibration 

An  environment  that  contains  a  greater  number  of  metal  objects  (such  as  inside  an  office 
building  with  metal  framed  walls  and  concrete  floors  containing  rebar)  than  would  normally  be 
encountered  during  the  use  of  a  metal  detector  outdoors  will  produce  an  elevated  level  of 
background  “noise.”  This  background  noise  may  completely  obscure  the  detector  responses  to 
low  metal  content  objects,  such  as  anti-personnel  mines.  In  order  to  compensate  for  this 
situation,  TMS  Master  includes  a  metal  detector  calibration  feature.  This  feature  allows  the 
calculation  of  an  offset  value  that  takes  into  account  the  measured  background  noise  level  at  a 
specific  location.  The  calibration  feature  may  be  initiated  at  any  time  prior  to  starting  a 
configured  exercise  by  selecting  the  “MD  Background  Cal...”  item  of  the  File  menu  during  an 
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Exercise  operation.  In  addition,  if  a  calibration  has  not  been  performed  prior  to  starting  an 
exercise  in  Demo  Mode,  the  user  will  be  prompted  to  do  so  when  metal  detector  data  is  first 
available.  In  this  case,  the  dialog  shown  in  Figure  B-7  is  first  presented  to  the  user. 


Figure  B-7.  Metal  Detector  Background  Calibration  prompt  dialog 

Selecting  Yes,  or  selecting  the  “MD  Background  Cal...”  File  menu  item,  will  open  a  dialog  that 
displays  the  current  received  metal  detector  data  necessary  for  calculating  the  background  noise 
calibration  value.  The  specific  contents  of  this  dialog  and  corresponding  calculation  method  are 
dependent  upon  the  type  of  mine  detector  used  in  the  exercise.  Figure  B-8  shows  the  dialog  for 
an  AN/PSS-12  detector.  This  dialog  displays  in  strip  chart  format  the  current  values  received 
from  the  FIU  representing  the  background  noise  level  measured  by  the  metal  detector.  The 
detector  operator  should  be  instructed  to  hold  the  detector  away  from  all  metal  objects  and  to  set 
the  metal  detector  sensitivity  as  low  as  possible  to  just  barely  indicate  a  response,  if  applicable. 
To  begin  the  calibration  process,  click  the  Start  button.  The  current  value  of  the  calculated  offset 
is  displayed  at  the  top  of  the  dialog.  Also  displayed  is  any  previously  calculated  value.  Once  the 
current  value  becomes  acceptably  steady,  click  the  Stop  button.  If  the  resulting  value  is 
considered  acceptable,  click  the  Send  button.  This  will  send  the  offset  value  to  the  FIU  for 
application  to  the  metal  detector.  The  metal  detector  output  should  then  be  reduced  by  the  value 
of  the  offset.  To  recalculate  the  offset  value,  click  the  Re-Start  button.  A  calibration  value  should 
be  sent  to  the  FIU  prior  to  starting  each  Demo  Mode  exercise.  To  reuse  a  value  for  all  subsequent 
exercises,  click  the  checkbox  at  the  bottom  of  the  dialog.  Doing  so  will  automatically  send  that 
value  to  the  FIU  at  the  beginning  of  subsequent  Demo  Mode  exercises.  If  the  reuse  option  is 
selected,  the  calibration  value  will  also  be  available  in  subsequent  TMS  execution  sessions.  If  the 
reuse  option  is  not  selected,  the  user  will  be  prompted  to  perfonn  background  calibration  at  the 
start  of  all  subsequent  exercises. 
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Figure  B-8.  AN/PSS-12  Detector  Background  Calibration  dialog. 

B.3.3  TMSMISP 

The  TMS  Mine  Interactive  Simulation  Program  (MISP)  application  includes  Demo  Mode 
capabilities  specific  to  generating  exercise  events  that  will  be  indicated  on  the  elevated  platform. 
Flowever,  there  are  no  unique  Demo  Mode  features  implemented  within  TMS  MISP  requiring  or 
involving  user  interaction. 

B.3.3. 1  Start  Up  for  Demo  Mode 

TMS  MISP  is  initialized  for  Demo  Mode  at  start  up  by  the  use  of  a  command  line 
parameter:  /d.  In  all  cases,  TMS  Master  launches  TMS  MISP.  Demo  Mode  configuration  of 
TMS  MISP  is  confirmed  by  opening  the  About  box  for  TMS  MISP  as  described  previously. 

B.3.3.2  Demo  Mode  Exercise  Events 

The  following  table  indicates  the  exercise  events  detected  by  TMS  MISP  and  the 
resulting  DU  light  behavior.  In  all  cases,  the  specified  light  behavior  will  occur  for 
approximately  seven  seconds.  At  the  end  of  this  period  the  lights  involved  will  be  commanded 
off. 
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Event 

Light  Behavior 

Real  Mine  Alarm 

Constant  green  light 

Virtual  Mine  Alarm 

Flashing  green  light 

Clutter  Alarm 

Constant  blue  light 

Clutter  Alarm  on 

Constant  blue  light  and  constant  (real  mine)  or  flashing 

Mine 

(virtual  mine)  green  light 

Near  Miss  Mine 

Constant  amber  light  and  constant  (real  mine)  or 

Alarm 

flashing  (virtual  mine)  green  light 

Far  Miss  Mine 
Alarm 

Constant  amber  light 

Mine  Detonation 

Constant  red  light  and  flashing  amber  light 

Table  B-l.  Demo  Mode  Exercise  Events  and  Corresponding  Light  Behavior 
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APPENDIX  J.  TMS  DETAILED  PROCEDURES 


TMS  Operator  Overview 

The  TMS  system  will  be  operated  and  maintained  on-site  by  members  of  the  ATC 

Geodetics  team.  These  individuals  are  highly  skilled  specialists  in  the  field  of  surveying  and 

geophysical  mapping.  Team  members  are  fluent  with  traditional  theadolite  as  well  as  modern 

RTK  GPS  methodologies.  The  geodetics  crew  will  set  up,  operate,  and  maintain  the  TMS 

system  for  daily  operations.  A  minimum  crew  of  two  members  will  be  required  for  this  test. 

TMS  Operator  Daily  Procedures 

1.  Start  generator,  and  allow  it  to  warm  up.  Prime  as  required.  Verily  Fuel  level  and  report  if 
fuel  will  be  required  for  day’s  operations.  Allow  voltage  and  frequency  to  maintain  220  V 
and  60  Hz,  respectively. 

2.  Unlock  data  van  and  turn  on  data  tenninal  power  center.  Allow  lights,  heating/cooling,  and 
all  auxiliary  hardware  to  power  on.  Initiate  the  UPS  from  the  standby.  Turn  on  the  console 
computers,  which  include  the  data  comins  server  and  workstations  1  and  2. 

3.  Set  up  outdoor  transmitter  equipment.  This  consists  of  four  ArcSecond  laser  transmitters. 
Transmitters  1  and  2  are  positioned  based  on  known  survey  control  points.  These  units  are 
then  turned  on,  and  operation  is  verified  by  a  rotating  turret.  This  allows  the  transmitters  to 
blanket  the  test  grid  with  a  3-D  laser  volume. 

4.  Power  up  FIU.  Operation  is  verified  by  a  red  light  emitting  diode  (LED)  contained  on  the 
FIU  on/off  switch.  Run  FIU  remotely  from  within  the  datacomms  server  using  “PC 
Anywhere”  remote  access  software.  Establish  basic  connection  from  the  FIU  to  the 
datacomms  server. 

5.  Run  “3Di  Workbench”  (software  is  housed  in  the  FIU)  and  begin  sensor  calibration.  Only 
one  crystal  may  be  used  for  the  calibration,  preferably  the  same  crystal  for  each  daily 
calibration  throughout  the  test.  Continue  calibration  by  choosing  six  random  positions  and 
two  observed  positions  over  a  known  baseline  length.  Perform  calibration  every  time 
transmitters  are  moved  (daily  before  test,  after  major  weather  delays). 

6.  Verify  3Di  software  captures  data  on  all  points  within  the  calibration  procedure  and 

completes  a  bundle  calculation.  Check  that  calibration  calculations  are  operating  within 
predetennined-allowable  errors.  If  the  error  is  within  the  desired  accuracy  range,  the 

software  passes  the  resulting  data  and  transmits  the  info  to  the  FIU.  If  the  error  exceeds 
allowable  standards,  the  calculation  will  fail  and  must  be  performed  again  until  a  satisfactory 
error  has  been  achieved. 

7.  Shut  down  Field  Instrumentation  Unit  (FIU).  Attach  all  sensors  and  cameras  to  FIU.  After 
all  sensors  have  been  connected  to  inputs  on  the  FIU,  the  FIU  is  restarted  by  pushing  the 
“on”  button  and  verifying  that  the  red  LED  indicator  is  on. 

8.  Re-establish  WI-FI  connection  with  “PC  Anywhere”.  Restart  3Di  workbench  software  and 
monitor  sensor  outputs  for  possible  laser  dropout.  This  can  be  caused  by  weather  or  blocking 
line-of-site. 

9.  Enable  (OPCLIENT)  in  DOS  prompt.  At  this  point,  the  system  is  ready  for  test  operator. 

10.  Assist  in  outfitting  test  subject  with  sensors  and  FIU  wagon.  Verify  after  outfitting  that  all 
sensors  are  operational. 
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11.  Prepare  for  first  lane  of  test.  Use  TMS  software  and  set  up  “mission  area”  for  specific  lane. 
Access  ground  truth  of  UXO  targets  and  paste  into  lane  area  within  TMS  and  CTMS.  Verify 
that  TMS  master  has  acquired  operator  near  first  lane.  Check  that  all  sensors  are  being 
acquired  and  no  blockage  has  occurred.  Confirm  video  stream  feed  is  operational. 

12.  Have  operator  place  detector  on  right  lane  boundary  marker.  Visually  check  that  detector  is 
in-fact,  above  marker  when  instructed  by  operator.  Manually  record  target  over  marker  as  a 
baseline  measurement. 

13.  Begin  recording  data  and  instruct  test  subject  to  begin  locating  UXO. 

14.  When  test  participant  declares  a  target,  manually  capture  alarm  by  declaring  a  target  within 
the  TMS  software.  Verity  target  was  accepted  and  saved  onto  real-time  mapping  of  grid  area 
(red  “x”).  Communicate  to  test  participant  that  run  may  continue. 

15.  Upon  completion  of  the  lane,  archive  telemetry  under  the  appropriate  operator,  date,  and 
time. 

16.  Upon  completion  of  test  grid  area,  survey  all  pin  flags  using  RTK  GPS  system.  Remove 
pin-flags  once  surveyed  and  return  to  data  van  for  reuse. 

Lane  1  Preparation. 

The  data  van  uses  the  TMS  software  at  this  point  to  set  up  a  mission  in  the  particular  lane 
or  location.  Ground  truth  of  the  location  or  lane  is  then  accessed  and  pulled  into  a  “run”.  The 
TMS  master  will  then  acquire  an  operator  in  the  field,  this  allows  you  to  see  on  screen  and  in  real 
time  the  location  of  the  operator’s  feet  and  the  detector  superimposed  over  the  lane  as  well  as  the 
location  of  potential  targets.  The  TMS  master  will  then  record  the  beginning  of  the  lane  right 
marker,  which  allows  for  a  QC  check  that  the  detector  is,  in-fact  where  the  tracking  system 
records.  The  alarm  is  verified  on  the  corner  reference  and  observed  to  be  directly  over  the  lane 
boundary  marker.  (Hence  the  note  in  the  TMS  alarm  files:  the  first  alarm  is  always  the 
beginning  of  the  lane).  The  GPS  alarms  file  only  contains  pin-flags. 

Logging. 

From  the  TMS  console,  the  mission  is  initiated  and  data  strings  are  saved,  as  well  as 
streaming  video. 

Encounters. 

When  an  operator  locates  a  possible  target  for  marking,  the  operator  communicates  to  the 
TMS  operator  via  radio  while  placing  the  head  of  the  detector  on  the  ground  directly  over  the 
target  to  be  marked.  The  TMS  operator  then  manually  presses  a  key  on  the  TMS  keyboard 
which  captures  the  alarm  location.  This  location  shows  up  as  a  distinct  mark  (red  x)  on  the  TMS 
as  well  as  the  CTMS  host.  Once  the  TMS  operator  verifies  that  the  target  has  been  saved,  a 
communication  is  made  to  the  test  subject  and  he/she  may  continue  the  run.  Upon  completion 
of  the  lane,  the  telemetry  is  archived  and  saved,  and  the  alarms  are  saved  under  the  appropriate 
operator/date/time  etc.  TMS  alarms  and  video  data  are  also  saved  at  this  point.  Saving  occurs 


J-2 


after  every  lane.  At  the  end  of  the  day,  all  pin-flags  are  surveyed  using  RTK  GPS  and  saved  as 
backups  to  the  TMS  alarms.  These  files  do  not  contain  the  beginning  right  boundary  maker  of 
each  lane. 

For  each  following  lane,  appropriate  ground  truth,  operator,  and  video  acquisition  must  be 
required.  This  process  is  estimated  to  have  the  duration  of  no  more  than  2  minutes. 
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CSTE-DTC-AT-AD-R 


16  January  2007 


MEMORANDUM  FOR  CHIEF,  Military  Environmental  Technology  Demonstration  Center 
Team,  ATTN:  Chris  Appelt 

SUBJECT:  Performance  Characteristics  and  Measurements  of  Operators  without  TMS 
Instrumentation,  Difference  using  0.5m  Halo  versus  1.0m  Halo. 


1.  Reference:  Analytical  Team  Report  06-AD A-026,  “Performance  Characteristics  and 
Measurements  of  Operators  without  TMS  Instrumentation”,  dated  May  2006. 

2.  The  same  data  from  the  2006  Operator  Perfonnance  Test  was  recalculated  using  a 
0.5-meter  radius  halo.  (If  a  finding  made  by  an  operator  was  within  a  0.5-meter  radius  halo  of 
the  ground  truth,  it  was  considered  a  hit.  Findings  outside  this  radius  were  considered  false 
alarms.  If  there  were  multiple  findings  within  a  halo,  only  one  was  counted  as  a  hit,  while  the 
others  were  not  considered  false  alarms.)  The  previous  report  used  a  radius  of  1.0-meters.  The 
recalculated  probabilities  of  detection  (Pd)  and  false  alann  rates  (FAR)  for  the  twelve  operators 
(five  experts,  five  novices,  and  two  quality  control  operators)  with  0.5m  halo,  along  with  the 
previously  calculated  Pd  and  FAR  with  1.0m  halo  are  presented  in  Table  1.  Plots  of  Pd  versus 
FAR  for  the  data  with  0.5m  halo  and  1.0m  halo  are  presented  in  Figures  1  and  2,  respectively. 

3.  The  Chi-Square  Test  for  differences  in  proportions  and  the  Mann- Whitney  test  were  used 
to  statistically  compare  the  performance  data  with  0.5m  halo  versus  data  with  1.0m  halo.  Using 
the  Chi-Square  Distribution  at  0.05  significance  level,  Pd  of  the  experts  and  novices  between 
0.5m  halo  and  1.0m  halo  were  not  found  to  be  significantly  different.  Using  the  Mann- Whitney 
Test  at  the  0.05  significance  level,  no  significant  differences  were  found  between  the  number  of 
false  alarms  between  the  0.5m  halo  and  1.0m  halo  of  either  the  novices  or  experts. 

4.  It  is  interesting  to  note  that  with  the  smaller  halo  size,  three  operators  that  previously  had 
100%  Pd,  now  do  not.  Also,  using  the  Mann- Whitney  test  at  the  0.05  significance  level,  there 
are  significantly  less  multiple  hits  with  the  smaller  halo  size. 

5.  This  memorandum  is  referenced  as  07-ADA-009,  and  the  point  of  contact  is  Selena 
Bednarz,  3-4528. 


REVIEWED  BY: 


APPROVED  BY: 


BARBARA  J.  GILLICH 
Technical  Lead,  Analytical  Team 
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NELLIE  M.  DUPREY 
Chief,  RAM/ILS  Engineering 
and  Analysis  Division 
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Table  1.  Summary  Table  of  Operators  Lanes  1-33  with  Uninstrumented  Data  with  0.5  m  Halo  and  1.0  m  Halo. 


0.5  m  halo 

1.0  m  halo 

DIFFERENCE 
(0.5m  halo  -  1.0m  halo) 

Point  Estimate 

False  Alarm 

No.  of 

Point  Estimate 

False  Alarm 

No.  of 

Point  Estimate 

False  Alarm 

No.  of 

Probability  of 

Rate, 

Multiple 

Probability  of 

Rate, 

Multiple 

Probability  of 

Rate, 

Multiple 

Operator 

detection  (Pd) 

cnts/sq  ma 

Hits 

detection  (Pd) 

cnts/sq  ma 

Hits 

detection  (Pd) 

cnts/sq  ma 

Hits 

E-1 

0.950 

0.178 

2 

1.000 

0.154 

26 

-0.050 

0.024 

-24  ! 

in 

E-2 

0.900 

0.077 

2 

0.917 

0.072 

7 

-0.017 

0.005 

-5 

r. 

<u 

E-3 

0.983 

0.098 

1 

1.000 

0.087 

12 

-0.017 

0.011 

-11 

Q. 

X 

E-4 

0.900 

0.039 

0 

0.933 

0.034 

4 

-0.033 

0.005 

-4 

LU 

E-5 

0.900 

0.099 

1 

0.933 

0.090 

9 

-0.033 

0.009 

-8 

Mean 

0.927 

0.098 

1.2 

0.957 

0.087 

11.6 

-0.030 

0.011 

-10.4  i 

N-1 

0.967 

0.072 

1 

1.000 

0.057 

16 

-0.033 

0.015 

-15  j 

</) 

N-2 

0.933 

0.036 

2 

0.983 

0.028 

8 

-0.050 

0.008 

-6  | 

0) 

o 

N-3 

0.917 

0.058 

8 

0.950 

0.047 

19 

-0.033 

0.011 

-11  i 

■> 

o 

N-4 

0.983 

0.191 

8 

0.983 

0.168 

34 

0.000 

0.023 

-26 

z 

N-5 

0.867 

0.022 

4 

0.933 

0.017 

6 

-0.067 

0.005 

-2  I 

Mean 

0.933 

0.076 

4.6 

0.970 

0.063 

16.6 

-0.037 

0.013 

-12.0  | 

Overall  Mean 

0.930 

0.087 

2.9 

0.963 

0.075 

14.1 

-0.033 

0.012 

-11.2 

o 

W-1 

0.917 

0.084 

4 

0.950 

0.073 

14 

-0.033 

0.011 

-10  I 

a 

W-2 

0.850 

0.072 

9 

0.917 

0.064 

14 

-0.067 

0.008 

-5 

length  x  width  of  Lanes  1-20  is  23.70m  x  1.5m,  Lanes  21-26  is  17.55m  x  1.5m,  Lanes  27-33  is  25.00m  x  1.5m 
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EXECUTIVE  SUMMARY 


The  U.S.  Army  Environmental  Center  (USAEC)  through  the  Environmental  Quality  Technology 
(EQT)  Program  requested  Aberdeen  Test  Center’s  (ATC)  Military  Environmental  Technology 
Demonstration  Center  (METDC)  to  develop  and  execute  a  plan  to  ascertain  and  document,  if  it 
exists,  a  level  of  influence  that  operators  may  have  on  unexploded  ordnance  (UXO)  detection 
technology  results.  The  primary  objective  of  the  test  is  to  determine  this  level  of  influence  and  to 
perform  an  analysis  of  operators'  detection  activities  to  identify  the  factors  that  produce 
variations  in  operator  performance. 

Until  recently,  the  UXO  and  Countermine  communities  both  have  relied  on  anecdotal  evidence 
to  account  for  the  widely  differing  levels  of  detection  achieved  from  various  operators. 
However,  recent  empirical  investigations  of  operator  influence  in  the  countermine  community 
have  discovered  substantial  variability  in  detection  perfonnance  between  operators  of  both 
currently  fielded  equipment  (AN19/PSS-12)  and  an  advanced  technology  then  under 
development  (HSATMIDS/PSS-14). 

To  date,  there  has  been  no  similar  attempt  to  define  objectively  the  level  of  operator  influence  in 
the  UXO  arena.  This  effort  seeks  to  determine  if  similar  individual  differences  in  operator 
performance  exist  and,  if  so,  to  identify  their  bases.  As  in  the  Countermine  work,  this  effort  will 
also  seek  an  explicit  description  of  the  human  factors  producing  any  differences  found.  Such  a 
description,  which  could  be  cast  as  a  cognitive  model,  holds  potential  to  serve  as  a  resource  for 
designing  operator  training  that  can  maximize  the  potential  of  fielded  UXO  detection  tools  and 
improve  detection. 

ATC  tested  10  geophysical  detector  operators  (5  novices  and  5  experts).  The  experts  had  more 
experience  with  geophysical  detection  than  did  the  novices. 

The  testing  indicated  anomalies  in  some  of  the  results  relating  to  expert  vs.  novice  perfonnance. 
The  overall  performance  of  the  novices  was  better  than  the  performance  of  the  experts.  The 
variability  of  the  novices’  probability  of  detection  (Pd)  results  was  less  affected  by  factors  such 
as  detector  head  height  and  velocity  than  the  variability  of  the  experts’  results.  In  addition,  PD 
was  affected  oppositely  by  detector  head  height  for  the  novices  vs.  the  experts.  These  results 
indicate  that  perhaps  periodic  refresher  training  would  be  beneficial  to  expert  operators  to 
improve  their  results  in  the  field. 
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PERFORMANCE  CHARACTERISTICS  AND  MEASUREMENTS 


Of  the  ten  operators  who  completed  the  test  grid,  five  were  classified  as  “Experts”  and  five  were 
classified  as  “Novices”  based  upon  their  experience.  Information  on  the  ten  operators  is 
provided  in  Table  1. 


Table  2:  Operator  Demographics 


Operator 

Age 

Race 

Gender 

Marital  Status 

Years  of 

Education 

Months  of 

EOD 

Experience 

Months  of 
UXO 

Experience 

Months  of 
Schonstadt 
Experience 

Prior  Military 
Experience 

Health 

Smoker 

(packs/day) 

Height 

(in) 

Weight 

(lbs) 

E-1 

34 

Caucasian 

Married 

12 

120 

48 

48 

Yes 

No 

72 

245 

E-2 

37 

Caucasian 

Male 

Divorced 

12 

156 

72 

72 

Yes 

Good 

No 

72 

260 

E-3 

28 

Caucasian 

Male 

Single 

13 

102 

96 

84 

Yes 

No 

69 

180 

E-4 

43 

Caucasian 

Male 

Married 

12 

252 

42 

42 

Yes 

No 

67 

200 

E-5 

25 

Other 

Single 

14 

78 

12 

12 

Yes 

No 

64 

130 

(A 

0) 

> 

o 

z 

N-1 

31 

Male 

Single 

16 

0 

0 

0.25 

No 

Good 

No 

67 

160 

N-2 

53 

Native  American 

Male 

Married 

16 

0 

0 

0 

No 

Good 

No 

73 

220 

N-3 

22 

Pacific  Islander 

Male 

Single 

12 

0 

0 

mmmm 

No 

■tfiSJMiS 

70 

223 

N-4 

40 

Caucasian 

Male 

Married 

12 

0 

0 

Yes 

No 

70 

230 

N-5 

24 

Pacific  Islander 

Male 

Single 

16 

0 

1.5 

1.5 

No 

Good 

No 

70 

210 

Using  the  Schonstadt  magnetometer,  each  operator  completed  33  lanes,  which  contained  a  total 
of  60  targets  buried  at  depths  ranging  from  six  to  30  inches.  Test  observers  maintained  a  daily 
log  to  record  test  data  and  conditions.  In  addition,  the  Schonstadt  was  equipped  with  two  sensors 
that  allowed  the  TMS  system  to  track  and  record  the  coordinates  of  the  sensors  at  a  rate  of  ten 
hertz.  Four  performance  characteristics  were  obtained  for  each  operator: 

1)  Lane  Velocity:  The  time  operators  took  to  complete  each  lane  was  manually 
recorded  on  the  daily  log.  Time  delays  due  to  equipment  issues  or  data  recording 
were  also  recorded  and  then  subtracted  from  the  total  lane  time.  This  “corrected” 
lane  length  was  then  divided  by  the  lane  time.  The  result  was  defined  as  the  Lane 
Velocity. 

2)  Percent  of  Lane  Area  Covered:  Using  the  TMS  data,  the  lateral  distance 
between  the  detector  head  and  each  point  on  a  0.25  meter  grid  within  the  lane  was 
calculated  for  each  recorded  coordinate  of  the  detector  head.  The  number  of 
points  on  the  grid  of  which  the  detector  head  came  within  0.25  meters  at  some 
point  during  the  run  was  divided  by  the  total  number  of  points  on  the  grid  and 
multiplied  by  100.  The  result  was  defined  as  the  Percent  of  Lane  Area  Covered. 

3)  Detector  Head  Height:  Using  the  TMS  data,  the  two  sensors’  positions  were 
used  to  calculate  a  vector  to  determine  the  position  of  the  detector  head.  The 
ground  altitude  at  the  nearest  surveyed  point  was  then  subtracted  from  the  altitude 
of  the  calculated  detector  head  position.  The  result  was  defined  as  the  Detector 
Head  Height. 

4)  Detector  Head  Velocity:  Using  the  TMS  data,  the  incremental  distance  traveled 
by  the  detector  head  was  calculated  by  taking  the  calculated  detector  head 
position  at  each  instance  and  subtracting  the  calculated  detector  head  position  at 
the  previous  instance.  The  incremental  distance  traveled  was  then  divided  by  the 
time  lapse  (normally  0.1  seconds).  The  result  was  defined  as  the  Detector  Head 
Velocity. 
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These  four  performance  characteristics  can  then  be  compared  to  the  two  perfonnance 
measurements  -  the  Probability  of  Detection  (Pd)  and  the  False  Alann  Rate  (FAR).  In  addition, 
a  third  performance  measurement  was  calculated  to  combine  both  the  Pd  and  FAR  and  facilitate 
comparisons  to  the  perfonnance  characteristics. 

1)  Probability  of  Detection  (Pd):  For  each  alarm  an  operator  noted  in  the  lane,  the 
distance  between  each  of  the  targets  in  the  field  and  the  alann  was  calculated.  If 
the  distance  was  less  than  one  meter,  then  the  target  was  considered  detected  no 
matter  in  which  lane  the  target  was  actually  located.  Multiple  detections  of  the 
same  target  were  ignored.  The  number  of  detected  targets  in  the  field  was  divided 
by  the  total  number  of  targets  in  the  field  (60  targets)  and  multiplied  by  100.  The 
result  was  defined  as  the  Pd. 

2)  False  Alarm  Rate  (FAR):  For  each  alarm  an  operator  noted  in  the  lane,  the 
distance  between  each  of  the  targets  in  the  field  and  the  alann  was  calculated.  If 
no  target  was  within  one  meter  of  the  alarm,  then  the  alarm  was  considered  a 
False  Alarm.  The  total  number  of  False  Alarms  was  divided  by  the  area  of  the 
field  (1131.5  square  meters).  The  result  was  defined  as  the  FAR. 

3)  Distance  Receiver  Operating  Characteristics  (ROC):  A  ROC  curve  is  an 
industry  standard  that  is  used  to  compare  the  performance  of  operators  and 
equipment  in  UXO  and  mine  detection.  It  consists  of  the  FAR  on  the  x-axis 
versus  the  Pd  on  the  y-axis.  Curves  nearer  the  upper  left-hand  comer  of  the  chart 
are  considered  better.  Therefore,  in  order  to  compare  the  operators’  perfonnance 
versus  each  of  the  characteristics,  the  Distance  ROC  was  calculated  as  the 
distance  from  the  upper  left-hand  corner  (coordinates  0,1)  that  an  operators’  point 
(FAR, Pd)  is  on  the  ROC  curve,  as  shown  in  the  following  equation: 


ast  _roc  =  J(\-Pd)2  +  (0-FAR)2 


RESULTS  OF  EXPERTS  VERSUS  NOVICES 


The  original  hypothesis  of  the  test  was  that  the  experts  would  perform  better  than  the  novices. 
The  performance  characteristics  of  the  experts  could  then  be  compared  to  the  novices  to 
determine  what  accounted  for  the  better  results.  However,  as  Figure  1  shows,  the  opposite 
results  were  observed. 
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Figure  1:  Dist  ROC  vs.  UXO  Experience 


Table  3  provides  a  summary  of  the  performance  measurements  for  each  of  the  operators,  as  well 
as  the  means  for  the  Novices  and  Experts  as  a  group. 


Table  3:  Performance  Measurements  of  Experts  and  Novices 


Operator 

Probability  of 
Detection  (Pd) 

False  Alarm 
Rate  (l/mA2) 

ROC 

Distance 

Experts 

E-l 

98.33% 

0.0477 

0.0506 

E-2 

83.33% 

0.0610 

0.1775 

E-3 

100.00% 

0.0875 

0.0875 

E-4 

61.67% 

0.0124 

0.3835 

E-5 

95.00% 

0.0963 

0.1085 

Mean 

87.67% 

0.0610 

0.1615 

Novices 

N-l 

98.33% 

0.0221 

0.0277 

N-2 

98.33% 

0.0133 

0.0213 

N-3 

95.00% 

0.0159 

0.0525 

N-4 

100.00% 

0.0468 

0.0468 

N-5 

100.00% 

0.0150 

0.0150 

Mean 

98.33% 

0.0226 

0.0327 

As  the  table  illustrates,  the  novice  group  perfonned  considerably  better  than  the  expert  group. 
The  lowest  novice  Pd  was  greater  than  or  equal  to  three  of  the  experts.  Only  one  expert  achieved 
a  lower  FAR  than  any  of  the  novices.  On  average,  this  produced  results  that,  when  plotted  on  a 
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standard  ROC  curve  as  shown  in  Figure  2,  were  five  times  closer  to  the  upper  left-hand  comer, 
indicating  superior  performance. 


In  order  to  understand  why  the  novices  performed  better  than  the  experts,  each  of  the  three 
performance  measurements  can  be  compared  individually  against  the  four  perfonnance 
characteristics  -  Detector  Head  Height,  Detector  Head  Velocity,  Lane  Velocity,  and  Percent  of 
Lane  Area  Covered.  Figure  3  through  Figure  6  compares  Pd  versus  the  four  characteristics, 
while  Figure  7  through  Figure  10  compares  FAR  versus  the  four  characteristics.  Figure  11 
through  Figure  14  compares  the  Distance  ROC  versus  the  four  characteristics. 


Generally,  the  results  as  a  whole  (independent  of  the  experts/novices  classification)  are  not 
surprising.  As  Figure  3  through  Figure  5  indicate,  the  novices’  performance  as  measured  by  Pd 
was  closely  grouped,  so  the  dependency  upon  the  performance  characteristics  is  difficult  to 
discern.  However,  for  the  experts,  Pd  performance  is  shown  to  suffer  as  the  height  and  velocity 
of  the  detector  head  increased.  In  addition,  though  the  correlation  was  not  as  significant,  Pd 
performance  also  decreased  as  the  lane  velocity  increased. 


As  Figure  7  through  Figure  9  show,  the  number  of  false  alarms  by  both  experts  and  novices’ 
generally  decreased  as  the  lane  velocity,  detector  head  height,  and  detector  head  velocity 
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increased.  However,  the  data  is  widely  scattered,  and  the  linear  regression  does  not  match  the 
data  very  closely  with  the  exception  of  the  data  for  the  novices  and  lane  velocity. 


Figure  3:  Experts  &  Novices  -  Pd  vs.  Detector  Head  Height 
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Figure  4:  Experts  &  Novices  -  Pd  vs.  Detector  Head  Velocity 


Figure  5:  Experts  &  Novices  -  Pd  vs.  Lane  Velocity 
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Figure  6:  Experts  &  Novices  -  Pd  vs.  Percent  of  Lane  Area  Covered 


Figure  7:  Experts  &  Novices  -  FAR  vs.  Detector  Head  Height 
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Figure  8:  Experts  &  Novices  -  FAR  vs.  Detector  Head  Velocity 


Figure  9:  Experts  &  Novices  -  FAR  vs.  Lane  Velocity 
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Figure  10:  Experts  &  Novices  -  FAR  vs.  Percent  of  Lane  Area  Covered 


Figure  11:  Experts  &  Novices  -  Dist  ROC  vs.  Detector  Head  Height 
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igure  12:  Experts  &  Novices  -  Dist  ROC  vs.  Detector  Head  Velocity 


Figure  13:  Experts  &  Novices  -  Dist  ROC  vs.  Lane  Velocity 
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Figure  14:  Experts  &  Novices  -  Dist  ROC  vs.  Percent  of 
Lane  Area  Covered 

Once  the  two  performance  measurements  are  combined  into  the  Distance  ROC,  differences 
between  the  experts  and  novices  begin  to  appear.  As  Figure  1 1  shows,  the  novices  performed 
better  as  the  detector  head  height  increased,  while  the  experts’  performance  deteriorated  as  the 
detector  head  height  increased.  This  should  only  be  true  for  the  novices  up  to  a  critical  detector 
head  height  value,  after  which  performance  should  decrease  due  to  reduced  received  signal 
strength.  A  similar  result  is  shown  in  Figure  12  and  Figure  13,  but  the  coefficients  of 
determination  (R  values)  do  not  indicate  a  close  lit  for  the  linear  regressions  with  the  exception 
of  the  line  for  the  experts  and  detector  head  velocity  in  Figure  12.  The  closer  the  R2  values  are  to 
1 .00,  the  greater  the  correlation  of  the  x  and  y  axis  data. 


Both  the  novices  and  the  experts  covered  the  lane  area  fairly  equally  with  little  variation.  This 
translated  into  no  significant  correlation  between  that  characteristic  and  Pd,  FAR,  and  Dist  ROC 
as  shown  Figure  6,  Figure  10,  and  Figure  14. 
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Table  4  summarizes  both  the  performance  measurements  and  characteristics  for  each  operator 
and  the  groups. 
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Table  4:  Summary  of  Data  for  Experts  and  Novices 


Operator 

Probability 

of 

Detection 

(Pd) 

False 

Alarm 

Rate 

(l/mA2) 

ROC 

Distance 

Lane 

Velocity 

(m/s) 

Percent 
of  Lane 
Area 
Covered 

Detector 

Head 

Height 

(in) 

Detector 

Head 

Velocity 

(m/s) 

Experts 

E-l 

98.33% 

0.0477 

0.0506 

0.09 

97.13% 

7.09 

1.38 

E-2 

83.33% 

0.0610 

0.1775 

0.07 

92.82% 

8.88 

1.74 

E-3 

100.00% 

0.0875 

0.0875 

0.07 

97.26% 

7.10 

1.73 

E-4 

61.67% 

0.0124 

0.3835 

0.10 

98.36% 

9.79 

2.62 

E-5 

95.00% 

0.0963 

0.1085 

0.06 

96.95% 

9.56 

1.72 

Mean 

87.67% 

0.0610 

0.1615 

0.08 

96.50% 

8.48 

1.84 

St.  Dev. 

0.159 

0.034 

0.132 

0.016 

0.021 

1.311 

0.463 

Novices 

N-l 

98.33% 

0.0221 

0.0277 

0.09 

95.82% 

8.24 

1.14 

N-2 

98.33% 

0.0133 

0.0213 

0.11 

97.03% 

7.86 

1.04 

N-3 

95.00% 

0.0159 

0.0525 

0.10 

97.03% 

6.61 

1.07 

N-4 

100.00% 

0.0468 

0.0468 

0.04 

97.52% 

6.52 

1.09 

N-5 

100.00% 

0.0150 

0.0150 

0.11 

96.62% 

9.27 

1.54 

Mean 

98.33% 

0.0226 

0.0327 

0.09 

96.80% 

7.70 

St.  Dev. 

0.020 

0.014 

0.016 

0.031 

0.006 

1.157 

0.208 

General 


observations 


from 
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Table  4: 


The  novices’  perfonnance  measurements  had  relatively  little  variation  between 
themselves  when  compared  to  the  experts’  perfonnance. 

The  novices  had  considerably  less  false  alarms  than  the  experts. 

There  was  less  variation  in  the  pace  at  which  the  expert  operators  completed  the 
lanes  as  compared  to  the  novices  (operator  N-4  skewed  the  novice  data  since  this 
person  was  between  two  and  three  times  slower  than  the  others). 

The  novices  tended  to  hold  the  detector  head  three  quarters  of  an  inch  lower  than 
the  experts,  on  the  average. 

The  novices  swung  the  detector  head  approximately  40  percent  slower  than  the 
experts. 


The  results  seem  to  indicate  that  the  position  and  speed  of  the  Schonstadt  detector  head  impact 
the  performance  measurements. 

Group  1  versus  Group  2 

Since  the  originally  hypothesis  proved  to  be  incorrect.  The  operators  were  grouped  based  upon 
their  performance  as  measured  by  Dist  ROC  rather  than  experience.  The  five  operators  with  the 
lowest  Dist  ROC  are  Group  1,  while  the  five  operators  with  the  highest  Dist  ROC  are  Group  2. 
Group  1  consists  of  four  novices  and  one  expert,  while  Group  2  consists  of  four  experts  and  one 
novice.  Table  5  summarizes  the  performance  measurements  for  the  two  groups. 


Table  5:  Performance  Measurements  of  Group  1  and  Group  2 


Operator 

Probability 
of  Detection 
(Pd) 

False  Alarm 

Rate 

(l/mA2) 

ROC 

Distance 

N-5 

100.00% 

0.0150 

0.0150 

N-2 

98.33% 

0.0133 

0.0213 

a 

a 

N-l 

98.33% 

0.0221 

0.0277 

© 

u 

N-4 

100.00% 

0.0468 

0.0468 

0 

E-l 

98.33% 

0.0477 

0.0506 

Mean 

99.00% 

0.0290 

0.0323 

N-3 

95.00% 

0.0159 

0.0525 

E-3 

100.00% 

0.0875 

0.0875 

a 

a 

E-5 

95.00% 

0.0963 

0.1085 

© 

u 

E-2 

83.33% 

0.0610 

0.1775 

o 

E-4 

61.67% 

0.0124 

0.3835 

Mean 

87.00% 

0.0546 

0.1619 

Reclassifying  the  operators  in  this  manner  resulted  in  further  widening  the  gap  in  the  mean  Pd 
measurement,  while  narrowing  the  gap  in  the  mean  FAR.  Only  one  operator  in  Group  2 
achieved  a  Pd  equal  to  or  greater  than  any  of  the  operators  in  Group  1 ,  while  two  operators  in 
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Group  2  achieved  a  FAR  less  than  the  average  FAR  in  Group  1 .  When  plotted  on  a  standard 
ROC  curve  as  shown  in  Figure  15,  Group  1  was  on  average  five  times  closer  to  the  upper  left- 
hand  corner  than  Group  2,  indicating  superior  performance. 


FAR  (1/mA2) 


Figure  15:  Group  1  vs.  Group  2  ROC  Curve 


Figure  16  through  Figure  19  compare  Pd  versus  the  four  characteristics,  while  Figure  20  through 
Figure  23  compare  FAR  versus  the  four  characteristics.  Figure  24  through  Figure  27  compare 
the  Distance  ROC  versus  the  four  characteristics. 

As  Figure  16  through  Figure  18  indicate,  Group  l’s  performance  as  measured  by  Pd  was  closely 
grouped,  so  the  dependency  upon  the  performance  characteristics  is  difficult  to  discern. 
However,  for  Group  2,  Pd  performance  is  again  shown  to  suffer  as  the  height  and  velocity  of  the 
detector  head  increased.  In  addition,  Pd  performance  also  decreased  as  the  lane  velocity 
increased.  The  graphs  are  very  similar  to  those  for  the  novices  and  experts  with  the  exception 
that  the  coefficient  of  detennination  for  the  regression  lines  deteriorated. 

As  Figure  20  through  Figure  22  show,  the  number  of  false  alarms  by  Group  1  generally 
decreased  as  the  lane  velocity,  detector  head  height,  and  detector  head  velocity  increased.  The 
same  held  true  for  Group  2  in  terms  of  detector  head  velocity  and  lane  velocity.  On  the  other 
hand,  the  number  of  false  alarms  by  Group  2  generally  increased  as  the  detector  head  height 
increased.  However,  the  data  is  widely  scattered,  and  the  linear  regression  does  not  match  the 
data  very  closely  with  the  exception  of  data  for  Group  1  and  the  detector  head  height. 
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Figure  16:  Group  1  &  Group  2  -  Pd  vs.  Detector  Head  Height 


Figure  17:  Group  1  &  Group  2  -  Pd  vs.  Detector  Head  Velocity 
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Figure  18:  Group  1  &  Group  2  -  Pd  vs.  Lane  Velocity 
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Figure  19:  Group  1  &  Group  2  -  Pd  vs.  Percent  of  Lane  Area  Covered 
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Figure  20:  Group  1  &  Group  2  -  FAR  vs.  Detector  Head  Height 


Figure  21:  Group  1  &  Group  2  -  FAR  vs.  Detector  Head  Velocity 
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Figure  22:  Group  1  &  Group  2  -  FAR  vs.  Lane  Velocity 


Figure  23:  Group  1  &  Group  2  -  FAR  vs.  Percent  of  Lane  Area  Covered 
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Figure  24:  Group  1  &  Group  2  -  Dist  ROC  vs.  Detector  Head  Height 


Figure  25:  Group  1  &  Group  2  -  Dist  ROC  vs.  Detector  Head  Velocity 
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Figure  26:  Group  1  &  Group  2  -  Dist  ROC  vs.  Lane  Velocity 


Figure  27:  Group  1  &  Group  2  -  Dist  ROC  vs.  Percent  of 
Lane  Area  Covered 

Again,  once  the  two  perfonnance  measurements  are  combined  into  the  Distance  ROC, 
differences  between  the  groups  begin  to  appear.  As  Figure  24  shows,  Group  1  performed  better 
as  the  detector  head  height  increased,  while  Group  2’s  perfonnance  deteriorated  as  the  detector 
head  height  increased.  A  similar  result  is  shown  in  Figure  25  and  Figure  26. 
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Both  groups  also  covered  the  lane  area  fairly  equally  with  little  variation.  This  translated  into  no 
significant  correlation  between  that  characteristic  and  Pd,  FAR,  and  Dist  ROC  as  shown  in 
Figure  19,  Figure  23,  and  Figure  27. 
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Table  6  summarizes  both  the  performance  measurements  and  characteristics  for  each  operator 
and  the  groups. 


L-28 


Table  6:  Summary  of  Data  for  Group  1  and  Group  2 


Operato 

r 

Probabilit 
y  of 

Detection 

(Pd) 

False 

Alarm 

Rate 

(l/mA2 

) 

ROC 

Distanc 

e 

Lane 
Velocit 
y  (m/s) 

Percent 
of  Lane 
Area 
Covere 
d 

Detecto 
r  Head 
Height 
(in) 

Detecto 

r  Head 
Velocit 
y  (m/s) 

N-5 

100.00% 

0.0150 

0.0150 

0.11 

96.62% 

9.27 

1.54 

N-2 

98.33% 

0.0133 

0.0213 

0.11 

97.03% 

7.86 

1.04 

& 

N-l 

98.33% 

0.0221 

0.0277 

0.09 

95.82% 

8.24 

1.14 

= 

o 

N-4 

100.00% 

0.0468 

0.0468 

0.04 

97.52% 

6.52 

1.09 

u 

u 

E-l 

98.33% 

0.0477 

0.0506 

0.09 

97.13% 

7.09 

1.38 

Mean 

99.00% 

0.0290 

0.0323 

0.09 

96.82% 

7.80 

1.24 

St.  Dev. 

0.009 

0.017 

0.016 

0.030 

0.006 

1.059 

0.214 

N-3 

95.00% 

0.0159 

0.0525 

0.10 

97.03% 

6.61 

1.07 

E-3 

100.00% 

0.0875 

0.0875 

0.07 

97.26% 

7.10 

1.73 

a. 

E-5 

95.00% 

0.0963 

0.1085 

0.06 

96.95% 

9.56 

1.72 

a 

o 

E-2 

83.33% 

0.0610 

0.1775 

0.07 

92.82% 

8.88 

1.74 

U 

E-4 

61.67% 

0.0124 

0.3835 

0.10 

98.36% 

9.79 

2.62 

Mean 

87.00% 

0.0546 

0.1619 

0.08 

96.48% 

8.39 

1.77 

St.  Dev. 

0.154 

0.039 

0.132 

0.017 

0.021 

1.450 

0.552 

General  observations  from 
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Table  6: 


■  The  lane  velocity  and  percent  area  covered  again  remained  close  between 
the  groups. 

■  As  seen  with  the  novices  and  experts,  the  better  performing  group  (in  this 
case  Group  1)  held  the  detector  head  lower  (~0.6  inches)  and  swung  it 
slower  (-30%). 

■  However,  Group  1  on  average  held  the  detector  head  slightly  higher  than 
the  novices  and  swung  it  slightly  faster. 

The  results  again  seem  to  indicate  that  the  position  and  speed  of  the  Schonstadt  detector  head 
impact  the  perfonnance  measurements. 

Group  1,  Group  2,  and  Group  3 

In  an  effort  to  further  highlight  differences,  the  data  for  the  operators  were  divided  into  three 
groups  based  upon  their  performance  as  measured  by  Dist  ROC.  The  three  operators  with  the 
lowest  Dist  ROC  are  Group  1;  the  three  operators  with  the  highest  Dist  ROC  are  Group  3;  while 
the  four  operators  with  the  middle  Dist  ROC  are  Group  2.  Group  1  consists  of  three  novices; 
Group  2  consists  of  two  novices  and  two  experts;  while  Group  3  consists  of  three  experts.  Table 
7  summarizes  the  perfonnance  measurements  for  the  three  groups. 


Table  7:  Performance  Summary  of  Group  1,  Group  2,  and  Group  3 


Operator 

Probability 

of 

Detection 

(Pd) 

False 

Alarm 

Rate 

(l/mA2) 

ROC 

Distance 

N-5 

100.00% 

0.0150 

0.0150 

Su 

3 

N-2 

98.33% 

0.0133 

0.0213 

o 

u 

N-l 

98.33% 

0.0221 

0.0277 

G 

Mean 

98.89% 

0.0168 

0.0213 

N-4 

100.00% 

0.0468 

0.0468 

fS 

a. 

E-l 

98.33% 

0.0477 

0.0506 

3 

O 

N-3 

95.00% 

0.0159 

0.0525 

u 

G 

E-3 

100.00% 

0.0875 

0.0875 

Mean 

98.33% 

0.0495 

0.0593 

E-5 

95.00% 

0.0963 

0.1085 

a 

3 

E-2 

83.33% 

0.0610 

0.1775 

O 

u 

E-4 

61.67% 

0.0124 

0.3835 

G 

Mean 

80.00% 

0.0566 

0.2232 

Reclassifying  the  operators  in  this  manner  resulted  in  further  widening  the  gap  between  the  best 
and  the  worst  performers  in  the  mean  Pd  and  FAR  measurements.  Only  one  operator  in  Group  3 
achieved  a  FAR  less  than  the  lowest  operator’s  FAR  in  Group  1 .  All  other  performance 
measurements  for  Group  1  were  notably  better  than  those  for  Group  2.  When  plotted  on  a 
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standard  ROC  curve  as  shown  in  Figure  28,  Group  1  was  on  average  more  than  ten  times  closer 
to  the  upper  left-hand  comer  than  Group  3,  indicating  superior  performance.  Group  2  was  on 
average  three  times  farther  from  the  upper  left-hand  corner  than  Group  1 ,  but  nearly  four  times 
closer  than  Group  3. 


FAR  (1/mA2) 


Figure  28:  Group  1  vs.  Group  2  vs.  Group  3  ROC  Curves 


Figure  29  through  Figure  32  compare  Pd  versus  the  four  characteristics,  while  Figure  33  through 
Figure  36  compare  FAR  versus  the  four  characteristics.  Figure  37  through  Figure  40  compare 
the  Distance  ROC  versus  the  four  characteristics. 


As  Figure  29  through  Figure  30  indicate,  Group  1  and  Group  2’s  data  indicates  that  perfonnance 
improved  as  the  detector  head  height  and  velocity  increased,  counter  to  that  observed  in  the 
previous  groupings.  On  the  other  hand,  for  Group  3,  Pd  performance  is  again  shown  to  suffer  as 
the  height  and  velocity  of  the  detector  head  increased.  However,  the  data  for  Group  3  and 
detector  head  height  is  widely  scattered.  In  addition,  Pd  performance  also  decreased  as  the  lane 
velocity  increased  for  all  three  groups  as  shown  in  Figure  31.  This  result  has  been  consistently 
observed. 


As  Figure  33  and  Figure  34  show,  the  data  for  FAR  versus  detector  head  height  and  velocity  is 
quite  dissimilar  for  each  group.  The  number  of  false  alarms  by  Group  1  generally  decreased 
slightly  as  the  detector  head  height  and  velocity  increased,  while  the  number  of  false  alarms  by 
Group  3  decreased  drastically  as  the  detector  head  height  and  velocity  increased.  Meanwhile,  the 
opposite  result  is  observed  for  Group  2.  The  number  of  false  alarms  increased  drastically  as  the 
detector  head  height  and  velocity  increased.  However,  large  variations  exist  in  the  data,  so  only 
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the  linear  regressions  for  Group  2  and  Group  3  for  FAR  versus  detector  head  velocity  are  good 
fits.  Again,  FAR  performance  also  improved  as  the  lane  velocity  increased  for  all  three  groups 
as  shown  in  Figure  35. 


Figure  29:  Groups  1-3  -  Pd  vs.  Detector  Head  Height 


Figure  30:  Groups  1-3  -  Pd  vs.  Detector  Head  Velocity 
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Figure  31:  Groups  1-3  -  Pd  vs.  Lane  Velocity 


Figure  32:  Groups  1-3  -  Pd  vs.  Percent  of  Lane  Area  Covered 
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Figure  33:  Groups  1-3  -  FAR  vs.  Detector  Head  Height 


Figure  34:  Groups  1-3  -  FAR  vs.  Detector  Head  Velocity 
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Figure  35:  Groups  1-3  -  FAR  vs.  Lane  Velocity 


Figure  36:  Groups  1-3  -  FAR  vs.  Percent  of  Lane  Area  Covered 
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Figure  37:  Groups  1-3  -  Dist  ROC  vs.  Detector  Head  Height 


Figure  38:  Groups  1-3  -  Dist  ROC  vs.  Detector  Head  Velocity 


L-36 


0.45 


Group  1 
Group  2 
Group  3 

-Linear  (Group  2) 
-Linear  (Group  1) 
-Linear  (Group  3) 


R2  =  0.7173 


0.06 

Lane  Velocity  (m/s) 


Figure  39:  Groups  1-3  -  Dist  ROC  vs.  Lane  Velocity 


Figure  40:  Groups  1-3  -  Dist  ROC  vs.  Percent  of  Lane  Area  Covered 

Again,  once  the  two  performance  measurements  are  combined  into  the  Distance  ROC,  greater 
differences  between  the  groups  can  be  observed.  In  Figure  37  and  Figure  38,  Group  2  and  Group 
3  show  improved  performance  as  detector  head  height  and  velocity  decrease,  while  Group  1 
shows  slightly  improved  performance  as  they  increase.  In  Figure  39,  Group  1  displays  better 
performance  as  the  lane  velocity  increases,  while  Group  2  and  Group  3  display  worse 
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performance  as  it  increases.  All  of  the  results  imply  that  there  may  be  a  middle  range  for  each 
performance  characteristic  that  leads  to  improved  perfonnance  measurements. 


All  three  groups  also  covered  the  lane  area  fairly  equally  with  little  variation.  This  translated 
into  no  significant  correlation  between  that  characteristic  and  Pd,  FAR,  and  Dist  ROC  as  shown 
in  Figure  32,  Figure  36,  and  Figure  40.  Table  8  summarizes  both  the  performance  measurements 
and  characteristics  for  each  operator  and  the  groups. 


Table  8:  Summary  of  Data  for  Groups  1-3 


Operato 

r 

Probabilit 
y  of 

Detection 

(Pd) 

False 

Alarm 

Rate 

(l/mA2 

) 

ROC 

Distanc 

e 

Lane 
Velocit 
y  (m/s) 

Percent 
of  Lane 
Area 
Covere 
d 

Detecto 
r  Head 
Height 
(in) 

Detecto 

r  Head 
Velocit 
y  (m/s) 

N-5 

100.00% 

0.0150 

0.0150 

0.11 

96.62% 

9.27 

1.54 

ft 

N-2 

98.33% 

0.0133 

0.0213 

0.11 

97.03% 

7.86 

1.04 

= 

o 

N-l 

98.33% 

0.0221 

0.0277 

0.09 

95.82% 

8.24 

1.14 

u 

g 

Mean 

98.89% 

0.0168 

0.0213 

0.10 

96.49% 

8.45 

1.24 

St.  Dev. 

0.010 

0.005 

0.006 

0.012 

0.006 

0.730 

0.265 

N-4 

100.00% 

0.0468 

0.0468 

0.04 

97.52% 

6.52 

1.09 

E-l 

98.33% 

0.0477 

0.0506 

0.09 

97.13% 

7.09 

1.38 

a 

3 

N-3 

95.00% 

0.0159 

0.0525 

0.10 

97.03% 

6.61 

1.07 

O 

u 

E-3 

100.00% 

0.0875 

0.0875 

0.07 

97.26% 

7.10 

1.73 

G 

Mean 

98.33% 

0.0495 

0.0593 

0.07 

97.23% 

6.83 

1.32 

St.  Dev. 

0.021 

0.029 

0.017 

0.011 

0.001 

0.237 

0.274 

E-5 

95.00% 

0.0963 

0.1085 

0.06 

96.95% 

9.56 

1.72 

m 

& 

E-2 

83.33% 

0.0610 

0.1775 

0.07 

92.82% 

8.88 

1.74 

3 

O 

E-4 

61.67% 

0.0124 

0.3835 

0.10 

98.36% 

9.79 

2.62 

u 

G 

Mean 

80.00% 

0.0566 

0.2232 

0.08 

96.05% 

9.41 

2.02 

St.  Dev. 

0.169 

0.042 

0.143 

0.021 

0.029 

0.476 

0.515 

General  observations  from  Table  8: 

■  The  average  lane  velocity  and  percent  area  covered  remained  close  between 
Group  1  and  Group  3  and  was  slightly  slower  than  in  the  previous  groupings; 
however,  Group  2  traveled  the  lane  at  a  faster  pace. 

■  As  seen  with  the  novices  and  experts,  the  better  performing  group  (in  this  case 
Group  1)  held  the  detector  head  lower  (~1  inch)  and  swung  it  slower  (-40%)  than 
worse  performing  group  (Group  3).  Interestingly,  Group  2  on  average  held  the 
detector  head  approximately  1.6  inches  lower  than  Group  1  and  swung  the 
detector  slightly  faster  (~6%). 
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The  results  again  seem  to  indicate  that  the  position  and  speed  of  the  Schonstadt  detector  head 
impact  the  performance  measurements.  Furthermore,  the  relationship  does  not  seem  to  be 
necessarily  linear,  since  Group  l’s  performance  measurements  were  better  with  a  mean  detector 
head  height  of  8.45  inches  as  compared  to  Group  2  and  Group  3’s  mean  of  6.83  and  9.41  inches, 
respectively. 


Detector  Head  Height  and  Velocity 


All  of  the  results  seem  to  indicate  that  the  position  and  speed  of  the  Schonstadt  detector  head 
impact  the  perfonnance  measurements.  To  investigate  this  further,  the  detector  head  height  and 
velocity  data  was  plotted  against  the  performance  measurements  without  the  novice,  experts,  and 
groups  classifications.  Since  the  range  of  mean  heights  and  velocities  of  the  best  and  worst 
groups  indicated  the  curves  may  be  parabolic,  either  a  linear  or  parabolic  regression  was  inserted 
to  fit  the  data  as  best  as  possible.  The  results  are  shown  in  Figure  41  through  Figure  46. 


Figure  43  indicates  that  the  relationship  between  the  detector  head  height  and  the  Dist  ROC  data 
may  be  approximated  by  a  parabolic  curve  with  the  better  Dist  ROC  measurements  achieved  in 
the  seven  to  eight  inches  range.  Breaking  the  Dist  ROC  measurement  into  its  individual  parts, 
Figure  41  reveals  that  the  detector  head  height  correlates  to  the  Pd  measurement  more  than  the 
FAR  measurement  as  shown  in  Figure  42.  Therefore,  this  suggests  that  an  operator  may  improve 
their  Pd  by  maintaining  the  Schonstadt  detector  head  between  seven  and  eight  inches  off  the 
ground;  however,  this  will  not  necessarily  improve  their  FAR. 


With  the  high  coefficient  of  determination,  the  parabolic  curve  in  Figure  46  fits  the  data  well. 
The  curve  suggests  that  the  best  performance  can  be  achieved  by  swinging  the  Schonstadt  so  that 
the  detector  head  is  traveling  at  a  velocity  between  one  and  1.25  meters  per  second.  The  curves 
in  Figure  44  and  Figure  45  imply  that  the  correlation  is  more  related  to  Pd  performance  than 
FAR  performance,  but  both  may  be  optimized  by  maintaining  a  velocity  in  this  range. 
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Figure  41:  Pd  vs.  Detector  Head  Height 
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Figure  42:  FAR  vs.  Detector  Head  Height 
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Figure  43:  Dist  ROC  vs.  Detector  Head  Height 


Figure  44:  Pd  vs.  Detector  Head  Velocity 
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Figure  45:  FAR  vs.  Detector  Head  Velocity 


Figure  46:  Dist  ROC  vs.  Detector  Head  Velocity  Ordnance  Type,  Depth,  and  Orientation 
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Five  different  ordnance  types  were  used  -  40imn,  60imn,  81mm,  105mm,  and  155mm.  Each 
type  was  buried  at  a  specific  depth  -  6,  12,  18,  24,  and  30  inches,  respectively.  The  ordnance 
were  placed  in  both  the  horizontal  and  vertical  orientations.  Table  9  provides  the  Pd 
measurements  for  each  ordnance  type,  depth,  and  orientation.  For  the  most  part,  ordnance  of  the 
same  type  were  more  likely  to  be  detected  when  oriented  vertically  rather  than  horizontally. 
However,  the  155mm  was  the  exception,  since  those  in  the  horizontal  orientation  had  a  higher 
Pd.  In  the  case  of  the  40mm  round,  both  orientations  had  a  100  percent  Pd,  while  the  81mm 
round  had  the  lowest  Pd  for  both  the  horizontal  and  vertical  orientations. 


Table  9:  Pd  by  Ordnance  Type,  Depth,  and  Orientation 


MUM 

Horizontal 

Vertical 

Total 

6 

100.0% 

100.0% 

100.0% 

12 

86.0% 

97.1% 

92.5% 

18 

83.3% 

85.7% 

84.6% 

105mm 

24 

90.0% 

100.0% 

96.2% 

155mm 

30 

96.7% 

91.4% 

93.8% 

Total 

90.8% 

94.7% 

92.7% 

Table  10  divides  the  data  into  the  expert  and  novice  groups.  As  the  table  shows,  the  novices 
achieved  a  perfect  100  percent  Pd  for  all  the  ordnance  in  the  vertical  orientation.  The  experts 
were  only  able  to  detect  the  155mm  round  in  the  horizontal  orientation  better  than  the  novices. 
The  experts  were  able  to  equal  the  novices’  performance  on  the  40mm  horizontal  and  vertical 
rounds  and  the  105mm  vertical  round.  On  average,  the  novices  achieved  Pd’s  ten  percentage 
points  higher  than  the  experts. 


Table  10:  Experts  &  Novices  -  Pd  by  Ordnance  Type  and  Orientation 


Type 

Novices 

Experts 

Total 

Horizontal 

40mm 

100.0% 

100.0% 

100.0% 

60mm 

92.0% 

80.0% 

86.0% 

81mm 

96.7% 

70.0% 

83.3% 

105mm 

100.0% 

80.0% 

90.0% 

155mm 

93.3% 

100.0% 

96.7% 

Total 

96.2% 

85.4% 

90.8% 

Vertical 

40mm 

100.0% 

100.0% 

100.0% 

60mm 

100.0% 

94.3% 

97.1% 

81mm 

100.0% 

71.4% 

85.7% 

105mm 

100.0% 

100.0% 

100.0% 

155mm 

100.0% 

82.9% 

91.4% 

Total 

100.0% 

89.4% 

94.7% 
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Table  11  divides  the  data  into  the  Group  1  and  Group  2  classifications.  As  the  table  shows, 
Group  1  achieved  a  perfect  100  percent  Pd  for  all  the  ordnance  except  the  60mm  horizontal  and 
the  81mm  vertical  rounds.  Group  2  did  not  detect  any  of  the  ordnance  types  better  than  Group  1; 
however,  Group  2  did  equal  Group  1  with  100  percent  Pd  for  the  40mm  horizontal  and  vertical 
and  105mm  vertical  rounds.  On  average,  Group  1  achieved  Pd’s  15  percentage  points  higher 
than  Group  2  for  the  horizontal  ordnance  and  nine  percentage  points  higher  for  the  vertical 
ordnance. 


Table  11:  Group  1  &  Group  2  -  Pd  by  Ordnance  Type  and  Orientation 


Type 

Group 

1 

Group 

2 

Total 

Horizontal 

40mm 

100.0% 

100.0% 

100.0% 

60mm 

92.0% 

80.0% 

86.0% 

81mm 

100.0% 

66.7% 

83.3% 

105mm 

100.0% 

80.0% 

90.0% 

155mm 

100.0% 

93.3% 

96.7% 

Total 

98.5% 

83.1% 

90.8% 

Vertical 

40mm 

100.0% 

100.0% 

100.0% 

60mm 

100.0% 

94.3% 

97.1% 

81mm 

97.1% 

74.3% 

85.7% 

105mm 

100.0% 

100.0% 

100.0% 

155mm 

100.0% 

82.9% 

91.4% 

Total 

99.4% 

90.0% 

94.7% 

Table  12  divides  the  data  into  the  Group  1,  Group  2,  and  Group  3  classifications.  As  the  table 
shows,  Group  1  achieved  a  perfect  100  percent  Pd  for  all  the  ordnance  except  the  60mm 
horizontal.  Group  2  achieved  a  better  Pd  for  the  60mm  horizontal  and  equaled  Groups  1  with 
100  percent  Pd  on  seven  others.  Group  3  realized  lower  Pd’s  than  both  Group  1  and  Group  2  on 
all  the  ordnance  types  with  the  exception  of  the  40mm  horizontal  and  vertical  and  the  105mm 
vertical,  which  all  operators  detected  100  percent.  On  average,  Group  1  achieved  Pd’s  22 
percentage  points  higher  than  Group  3  for  the  horizontal  ordnance  and  17  percentage  points 
higher  for  the  vertical  ordnance.  However,  Group  1  only  realized  Pd’s  slightly  higher  Pd’s  than 
Group  2  for  both  orientations. 
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Table  12:  Groups  1,  2,  &  3  -  Pd  By  Ordnance  Type  and  Orientation 


Type 

Group 

1 

Group 

2 

Group 

3 

Total 

Horizontal 

40mm 

100.0% 

100.0% 

100.0% 

100.0% 

60mm 

86.7% 

100.0% 

66.7% 

86.0% 

81mm 

100.0% 

95.8% 

50.0% 

83.3% 

105mm 

100.0% 

100.0% 

66.7% 

90.0% 

155mm 

100.0% 

91.7% 

100.0% 

96.7% 

Total 

97.4% 

97.1% 

75.6% 

90.8% 

Vertical 

40mm 

100.0% 

100.0% 

100.0% 

100.0% 

60mm 

100.0% 

100.0% 

90.5% 

97.1% 

81mm 

100.0% 

96.4% 

57.1% 

85.7% 

105mm 

100.0% 

100.0% 

100.0% 

100.0% 

155mm 

100.0% 

100.0% 

71.4% 

91.4% 

Total 

100.0% 

99.3% 

83.3% 

94.7% 

In  summary,  the  data  indicates  that  the  60mm  horizontal  and  the  81mm  in  both  orientations 
proved  to  be  the  most  difficult  ordnance  types  to  locate.  The  superior  performances  by  the 
novices  and  Group  1  can  be  attributed  to  their  perfonnance  on  these  ordnance  types  and 
orientations.  Table  13  provides  the  difference  between  the  Pd  of  the  best  and  worst  group  for 
each  of  the  three  classifications  for  these  ordnance  types  and  orientations. 


Table  13:  Pd  Differences 


60mm 

Horizontal 

81mm 

Horizontal 

81mm 

Vertical 

Novices  vs.  Experts 

26.7% 

28.6% 

Group  1  vs.  Group  2 

33.3% 

Group  1  vs.  Group  3 

42.9%  ; 

L-45 


2.0  CONCLUSIONS 


Contrary  to  the  original  hypothesis  of  the  experiment,  the  operators  classified  as  Novices 
achieved  better  performance  results  than  those  classified  as  Experts.  After  a  review  of  the  data, 
the  following  characteristics  were  observed: 


■  Both  the  novices  and  the  experts  covered  the  lane  area  fairly  equally  with  little 
variation. 

■  The  novices  had  considerably  less  false  alarms  than  the  experts. 

■  The  novices  tended  to  hold  the  detector  head  three  quarters  of  an  inch  lower  than 
the  experts. 

■  The  novices  swung  the  detector  head  approximately  40  percent  slower  than  the 
experts. 

The  results  seem  to  indicate  that  the  position  and  speed  of  the  Schonstadt  detector  head  impact 
the  performance  measurements. 


When  divided  into  two  groups  based  upon  their  performance  measurements,  Pd  performance  is 
again  shown  to  suffer  as  the  height  and  velocity  of  the  detector  head  increased.  In  addition,  Pd 
performance  also  decreased  as  the  lane  velocity  increased.  The  number  of  false  alarms  generally 
decreased  as  the  lane  velocity  and  detector  head  velocity  increased.  The  better  perfonning  group 
(in  this  case  Group  1)  held  the  detector  head  lower  (~0.6  inches)  and  swung  it  slower  (-30%). 
However,  Group  1  perfonned  better  as  the  detector  head  height  increased,  while  Group  2’s 
performance  deteriorated  as  the  detector  head  height  increased.  The  results  again  seem  to 
indicate  that  the  position  and  speed  of  the  Schonstadt  detector  head  impact  the  performance 
measurements 


When  divided  into  three  groups  based  upon  their  performance  measurements,  Group  1  and 
Group  2’s  data  indicates  that  perfonnance  improved  as  the  detector  head  height  and  velocity 
increased,  counter  to  that  observed  in  the  previous  groupings.  On  the  other  hand,  for  Group  3, 
Pd  performance  is  again  shown  to  suffer  as  the  height  and  velocity  of  the  detector  head 
increased.  The  number  of  false  alarms  by  Group  1  generally  decreased  slightly  as  the  detector 
head  height  and  velocity  increased,  while  the  number  of  false  alanns  by  Group  3  decreased 
drastically  as  the  detector  head  height  and  velocity  increased.  Meanwhile,  the  opposite  result  is 
observed  for  Group  2.  The  number  of  false  alarms  increased  drastically  as  the  detector  head 
height  and  velocity  increased.  As  seen  with  the  novices  and  experts,  the  better  performing  group 
(in  this  case  Group  1)  held  the  detector  head  lower  (-1  inch)  and  swung  it  slower  (-40%)  than 
worse  performing  group  (Group  3).  Interestingly,  Group  2  on  average  held  the  detector  head 
approximately  1.6  inches  lower  than  Group  1  and  swung  the  detector  slightly  faster  (-6%).  All 
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of  the  results  imply  that  there  may  be  a  middle  range  for  each  performance  characteristic  that 
leads  to  improved  performance  measurements. 


Due  to  the  many  uncontrolled  variables  in  this  experiment,  no  firm  conclusion  can  be  drawn 
about  these  perfonnance  characteristics.  However,  the  data  suggests  that  operators  who  maintain 
the  detector  head  seven  to  eight  inches  from  the  ground  while  swing  the  detector  head  at  a 
velocity  between  one  and  1 .25  meters  per  second  will  have  a  better  perfonnance  than  those  who 
have  performance  characteristics  outside  of  this  range. 


L-47 


APPENDIX  M.  HRED  STRESS  AND  WORKLOAD  ASSESSMENT  SUMMARY 
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Background 


The  U.  S.  Army  Research  Laboratory  (ARL)  has  developed  a  stress  assessment  battery, 
Stress  Assessment  and  Monitoring  System  (SAMS)  that  measures  individual  reactions  to 
both  physical  and  mental  stressors  using  non-invasive  methods  that  cause  little  or  no 
interference  to  the  performance  of  a  variety  of  tasks.  This  battery  of  psychological 
questionnaires  has  proved  its  sensitivity  to  the  degree  of  stress  experienced  in  a  variety  of 
situations  and  includes  several  standardized  measures  that  have  demonstrated  construct 
validity  within  the  stress  research  literature.  SAMS  uses  a  standardized  methodology  that 
has  been  validated  using  military  and  civilian  populations.  The  battery  provides  overall 
measures  of  stress  and  subcomponents  of  stress.  Identifying  subcomponents  related  to  stress 
provides  insight  concerning  why  the  individual  is  experiencing  stress  and  possible  solutions 
to  alleviate  the  stress. 

Salivary  amylase  concentrations  are  predictive  of  plasma  catecholamine  levels  and  can  be 
used  as  a  measure  of  stress  (Chatterton,  Vogelsong,  Lu,  Ellman,  &  Hudgens,  1996).  Salivary 
amylase  and  cortisol  provide  a  non  invasive,  objective,  physiological  measure  of  overall 
stress. 


The  objective  of  this  field  experiment  was  to  provide  quantifiable  measures  of  the 
psychological  and  physiological  stress  induced  by  UXO  clearance  operations. 

Methods 


Participants 

Test  participants  were  10  civilian  employees,  5  experts  and  5  student  novices.  Experts 
consisted  of  4  males  and  1  female  and  had  an  average  age  of  33.4  years  (SD=7.2  years)  with  an 
average  education  of  12.6  years  (SD=0.9  years).  All  of  the  experts  had  prior  military  experience. 
Experts  had  a  mean  of  11.8  years  (SD=5.7  years)  of  EOD  experience  and  4.5  years  (SD=2.6 
years)  of  UXO  experience,  with  a  mean  of  4.3  years  (SD=2.3  years)  using  the  Schonstedt  sensor. 
Student  novices  were  all  males  with  an  average  age  of  34  years  (SD=12.7  years)  and  an  average 
education  level  of  14.4  years  (SD=.9  years)  One  student  novice  had  prior  military  experience. 

Apparatus 

Batteries  of  standardized  psychological  trait  and  state  questionnaires  were  used  in 
conjunction  with  noninvasive  physiological  stress  measures,  salivary  amylase  and  salivary 
cortisol.  The  battery  of  psychological  questionnaires  has  proved  its  sensitivity  to  the  degree  of 
stress  experienced  in  a  variety  of  situations  and  includes  several  standardized  measures  that  have 
demonstrated  construct  validity  within  the  stress  research  literature.  These  measures  can  be 
selected  and  tailored  to  the  research  objectives  and  experimental  design  as  needed.  This  stress 
assessment  and  monitoring  system  uses  a  standardized  methodology  that  has  been  validated 
using  both  military  and  civilian  populations. 

General  information  Questionnaire:  This  questionnaire  includes  general  demographic 
information  (age,  education,  job  title,  etc.)  and  questions  regarding  the  participant’s  health  status. 
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Psychological  Trait  Measures:  The  following  trait  measure  questionnaires  are  used  to  assess 
individual  characteristics,  and  were  administered  on  a  non-test  day. 

Multiple  Affect  Adjective  Check  List  -  Revised  (MAACL-R;  Zuckerman  &  Lubin,  1985). 
The  General  form  of  the  MAACL-R  has  five  primary  subscales  (Anxiety,  Depression,  Hostility, 
Positive  Affect,  and  Sensation  Seeking)  derived  from  a  one-page  list  of  132  adjectives.  The 
participants  check  all  the  words  that  describe  how  they  “generally”  feel.  An  overall  distress 
score,  Dysphoria  or  Negative  Affect  is  calculated  using  the  Anxiety,  Depression,  and  Hostility 
scores.  Positive  Affect  measures  a  sense  of  well  being  and  scores  vary  independently  from 
Negative  Affect  scores. 

Psychological  State  Measures.  A  five-minute  battery  of  stress  perception  measures  were 
administered  at  strategic  time  points  before,  during,  and  after  the  different  conditions  being 
measured.  The  following  state  measures  are  included: 

Specific  Rating  of  Events  Scale  (SRE;  Fatkin,  King,  &  Hudgens,  1990).  The  SRE  allows  the 
participants  to  rate  (on  a  scale  of  0-100)  how  stressed  they  felt  during  a  specified  time  period. 

Subjective  Stress  Scale  (SUBJ;  Kerle  &  Bialek,  1958).  This  scale  detects  significant 
affective  changes  in  stressful  conditions.  Participants  are  instructed  to  select  one  word  from  a 
list  of  15  adjectives  that  best  describes  how  they  felt  during  a  specified  time. 

Multiple  Affect  Adjective  Check  List  -  Revised,  Today  form  (MAACL-R;  Zuckerman  & 
Lubin,  1985).  The  MAACL-R  Today  form  is  used  to  examine  changes  in  specific  affects  in 
response  to  stressful  situations.  This  measure  is  identical  to  the  General  fonn,  except  the 
participants  are  instructed  to  answer  according  to  how  they  felt  during  a  specified  time  period. 

Self-Efficacy  Scale.  The  Self-Efficacy  Scale  (SES;  Bandura,  1977;  Hudgens,  Malto,  Geddie, 
&  Fatkin,  1991;  Sherer  et  ah,  1982)  asks  respondents  to  rate  their  level  of  confidence  in  their 
ability  to  do  well  with  reference  to  anticipation  of  their  mission  in  detecting  UXO.  Positive 
correlations  have  been  obtained  between  self-efficacy  and  vocational,  educational  and  military 
success.  This  questionnaire  was  given  once  with  the  baseline  assessment. 

Workload  Assessment:  National  Aeronautics  and  Space  Administration-Task  Load  Index 
(NASA-TLX).  The  NASA  TLX  is  a  subjective  measure  of  workload  developed  by  the  Human 
Performance  group  at  NASA  Ames  Research  Center.  Scores  include  a  total  overall  workload 
score  based  on  the  weighted  average  of  ratings  on  six  subscales:  mental  demand,  physical 
demand,  temporal  demand,  own  performance,  effort  level,  and  frustration  level. 

Physiological  Assessment:  Amylase  is  an  enzyme  that  hydrolyzes  starch  to  oligosaccharides 
and  then  slowly  changes  to  maltose  and  glucose.  Salivary  amylase  concentrations  are  predictive 
of  plasma  catecholamine  levels  and  can  be  used  as  a  measure  of  stress  (Chatterton,  Vogelsong, 
Lu,  Ellman,  &  Hudgens,  1996).  Measurement  of  amylase  concentration  in  saliva  includes  the 
observation  of  chemical  color  changes  according  to  standard  photometric  procedures  developed 
by  Northwestern  University  (Chatterton  et  ah,  1996).  Salivary  cortisol  was  assayed  using 
radioimmunoassay.  Amylase  and  salivary  cortisol  were  assayed  at  Northwestern  University. 

Procedure 

Trait  measures,  baseline  state  measures,  and  baseline  amylase  and  salivary  cortisol  data 
were  collected  on  a  non-test  day  prior  to  the  study.  Trait  and  baseline  data  collection  required 
approximately  one  hour.  State  data  collection  sessions  required  approximately  five  minutes.  Pre 
measures  for  the  state  stress  questionnaire  data  and  salivary  amylase  and  cortisol  were  collected 
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in  the  morning  prior  to  testing.  Three  during  measures  were  collected  at  given  points.  Table  1 
identifies  the  UXO  detection  lanes  each  participant  completed  for  each  during  condition.  Post 
measures  for  the  stress  questionnaire  data  and  saliva  were  collected  at  the  end  of  each  test  day. 
For  during  and  post  sessions  participants’  were  instructed  to  complete  during  and  post  surveys 
according  to  ‘how  they  felt  during  the  UXO  task  since  the  last  time  they  completed  the 
questionnaires.’ 

The  stability  of  the  saliva  sample  requires  it  be  kept  cold.  Saliva  samples  were  stored  in 
coolers  containing  ice  packs  immediately  following  collection.  At  the  end  of  the  day  saliva 
samples  were  placed  in  a  freezer  and  stored  until  they  were  shipped  to  Northwestern  for  assay. 
Saliva  was  packed  in  coolers  with  dry  ice  and  shipped  via  overnight  delivery. 

Experimental  Design 

This  research  used  2  groups  (Experts  vs.  Novices)  x  2  conditions  (Schonstedt  with  TMS 
instrumentation  vs.  Schonstedt  without  TMS  instrumentation)  x  6  sessions  (baseline,  pre, 
during  1,  during  2,  during  3,  post)  design  with  groups  as  a  between-subjects  variable  and 
conditions  and  sessions  as  within-subjects  variables.  Dependent  measures  were  the  scores  from 
the  stress  assessments,  NASA  TLX  workload  assessments,  and  saliva  assays  for  cortisol  and 
amylase. 

Results  and  Discussion 

Data  analysis  is  limited  to  descriptive  techniques  due  to  the  small  group  size.  Stress 
assessment  graphs  are  presented  in  Appendix  B.  Results  from  the  post  MAACL-R  are  plotted 
against  comparison  data  from  other  studies.  Comparison  charts  are  in  Appendix  C. 

Psychological  Trait  and  State  Measures 

Self  Efficacy  Scale  (SSE) 

The  SSE  asks  participants  to  rate  how  confident  they  are  in  their  ability  to  accomplish  the 
upcoming  tasks  on  a  scale  of  1-10.  This  measure  is  collected  once  during  administration  of 
baseline  measures.  Two  novices  did  not  complete  this  fonn.  All  participants  that  responded  had 
a  high  level  of  confidence  on  this  measure.  Three  experts  responded  with  a  10,  one  with  9.5  and 
one  with  9.0.  The  three  novices  responded  with  a  10,  9,  and  8. 

Specific  Rating  of  Events 

The  SRE  asks  participants  to  rate  how  much  stress  they  feel  on  a  scale  of  1-100.  It 
represents  an  overall  measure  of  stress.  Experts  did  not  experience  much  variation  from 
baseline.  Novices  reported  higher  SRE  when  compared  to  experts.  Trait  and  baseline  SRE 
scores  were  higher  for  novices  and  measures  taken  during  testing  did  not  vary  above  novices’ 
trait  measure. 

Subjective  Stress 

This  scale  detects  significant  affective  changes  in  stressful  conditions.  Participants  are 
instructed  to  select  one  word  from  a  list  of  15  adjectives  that  best  describes  how  they  felt  during 
a  specified  time.  Like  the  SRE  this  measure  represents  an  overall  measure  of  stress.  Experts’ 
scores  increased  slightly  above  baseline  during  UXO  test  sessions  with  the  added 
instrumentation.  Their  scores  did  not  increase  during  testing  without  the  added  instrumentation. 
Novices’  scores  were  generally  higher  then  experts. 
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Their  scores  increased  above  baseline  during  UXO  test  sessions  with  added  instrumentation. 
Their  scores  increased  for  the  without  instrumentation  condition  during  the  third  session  and  post 
session. 

MAACL-R 

Five  subscale  scores  for  the  MAACL-R,  anxiety,  depression,  hostility,  positive  affect,  and 
dysphoria  (a  composite  measure  that  combines  anxiety,  depression,  and  hostility  scores  and 
represents  negative  affect)  are  used  in  the  stress  assessments.  MAACL-R  scores  are  used  as  an 
indication  of  a  person’s  affect  or  response  to  an  event,  not  as  clinical  indicators.  These  scores 
represent  subcomponents  of  stress  and  further  refine  the  stress  assessments  by  giving  an 
indication  of  what  may  be  causing  the  psychological  stress  to  occur. 

The  MAACL-R  anxiety  scale  is  associated  with  anticipation  or  uncertainty  associated 
with  an  event.  Experts’  anxiety  levels  remained  at  baseline  levels  throughout  test  sessions. 
Novices’  levels  of  anxiety  increased  for  the  during  sessions.  They  had  a  larger  increase  when 
wearing  the  additional  TMS  instrumentation.  It  appears  adding  the  unfamiliar  TMS 
instrumentation  increased  the  uncertainty  for  novices.  Experts  were  more  comfortable  with  the 
added  equipment.  It  may  be  that  inexperienced  users  had  difficulty  adjusting  to  the  changes 
associated  with  additional  new  equipment. 

The  MAACL-R  depression  scale  is  a  measure  of  the  individual’s  sense  of  failure  or 
ceaseless  striving.  It  often  correlates  with  measures  of  morale  and  cohesion.  Experts’ 
depression  measures  remained  around  baseline  throughout  test  sessions.  Novices’  showed  a 
slight  increase  in  depression  during  testing  with  the  added  TMS  instrumentation.  They  had  a 
large  increase  for  the  pre  and  the  first  during  session  without  instrumentation.  This  may  indicate 
some  sense  of  concern  or  dissatisfaction  associated  with  their  perfonnance  during  testing. 

The  MAACL-R  hostility  scale  represents  frustration  with  the  task,  equipment,  or 
performance.  Experts’  hostility  scores  remained  around  baseline.  Novices’  scores  increased 
with  the  added  TMS  instrumentation  during  testing.  They  showed  large  increases  for  the  pre  and 
first  during  session  for  the  no  instrument  condition.  This  may  indicate  novices  were  frustrated 
with  the  equipment  or  their  performance  using  the  equipment. 

The  MAACL-R  dysphoria  is  a  composite  score  of  anxiety,  depression,  and  hostility.  It  is 
used  as  a  measure  of  overall  negative  affect.  It  followed  the  trends  already  discussed  for  the 
anxiety,  depression,  and  hostility  measures.  Experts’  dysphoria  scores  remained  around 
baseline.  Novices’  scores  increased  during  testing  with  the  added  TMS  equipment  and  for  the 
pre  and  first  test  session  without  added  instrumentation. 

The  MAACL-R  positive  affect  score  measures  an  overall  sense  of  well  being.  While 
positive  affect  is  generally  expected  to  vary  in  an  opposite  direction  from  negative  affect  the  two 
subscale  measures  represent  different  dimensions  and  may  vary  independently  during  sessions. 
For  positive  affect  experts  and  novices  showed  slight  variations  from  baseline  throughout  testing. 

Physiological  Measures  of  Stress:  Amylase  and  Salivary  Cortisol 

Participants  provided  saliva  samples  when  they  completed  questionnaires  for  baseline, 
pre,  duringl,  during  2,  during  3,  and  post  sessions.  Saliva  samples  collected  during  testing  were 
shipped  on  dry  ice  to  Northwestern  University  and  assayed  for  amylase  and  salivary  cortisol. 
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Amylase  and  cortisol  represent  overall  measures  of  stress.  Environmental,  physical,  or 
psychological  stressors  may  cause  elevated  levels  of  amylase  or  cortisol. 

Salivary  amylase  is  an  enzyme  secreted  in  response  to  sympathetic  nervous  system 
activity.  It  is  used  as  a  non-invasive  measure  representative  of  the  individual’s  catecholamine 
response.  Amylase  activity  of  400U/ml  or  greater  represent  moderate  to  high  levels  of  stress 
response  (Chatterton  et  ah,  1996).  Average  amylase  scores  over  all  sessions  and  conditions 
ranged  from  135.6  U/ml  (SD=79.6)  through  262.7  U/ml  (SD=176.3).  This  represents  low  levels 
of  stress.  Experts’  amylase  levels  remained  around  baseline  for  the  no  instrument  condition, 
with  slight  elevations  for  the  third  during  session  and  post  session.  Their  amylase  levels 
increased  during  the  UXO  task  with  the  additional  TMS  instrumentation.  This  indicates  the 
additional  instrumentation  caused  some  minor  problems  for  the  experts.  Experts’  scores  on  the 
MAACL-R  remained  around  baseline.  SRE  and  SUBJ  increased  slightly  during  UXO  tasks. 
The  lack  of  change  in  experts’  MAACL-R  scores  indicates  this  slight  elevation  in  stress  may  be 
due  to  physical  factors  rather  then  psychological  factors. 

With  the  exception  of  baseline  and  the  second  during  test  session  with  instrumentation, 
novices  had  higher  amylase  levels  then  experts.  For  both  conditions  amylase  levels  for  novices 
were  above  baseline  across  all  sessions.  Novices’  MAACL-R  scores  generally  increased  above 
baseline  during  the  UXO  tasks.  Psychological  and  physical  stressors  most  likely  contribute  to 
increased  amylase  activity  for  novices. 

Cortisol  is  an  adrenal  honnone  often  associated  with  stress.  Salivary  amylase  increases 
more  rapidly  then  cortisol  during  a  stressful  event.  Salivary  cortisol  was  assayed  at 
Northwestern  University  using  radioimmunoassay  without  extraction.  Salivary  cortisol 
increased  slightly  above  baseline  for  the  experts’  pre  measure  without  instrumentation.  Novices’ 
cortisol  increased  slightly  for  the  first  during  and  post  session  with  instrumentation.  Salivary 
cortisol  measures  were  low  to  moderate  for  both  groups. 

Stress  Assessments  and  UXO  Detection  Performance 

Correlations  were  computed  between  stress  measures  and  the  operator  perfonnance 
measures,  probability  of  detection  (PD),  and  background  alarm  rate  (BAR).  PD  equals  the 
number  of  declared  targets  divided  by  the  number  of  actual  targets.  Each  target  had  an 
imaginary  1  meter  safe-halo.  BAR  refers  to  the  number  of  false  declarations  per  square  meter 
and  is  a  measure  of  effectiveness  or  efficiency. 

Due  to  the  small  group  size  correlations  were  computed  by  combining  the  experts  and  novices. 

Correlations  were  computed  between  PD  with  the  MAACL-R  subscales  for  anxiety, 
depression,  and  hostility,  and  subjective  stress,  specific  rating  of  events,  amylase  and  salivary 
cortisol  (Table2).  There  was  a  significant  correlation  between  positive  affect  and  performance  in 
the  post  Schonstedt  with  no  instrument  condition. 

Correlations  were  computed  between  the  BAR  and  the  MAACL-R  subscales  for  anxiety, 
depression,  and  hostility,  subjective  stress,  specific  rating  of  events,  amylase  and  salivary  cortisol 
(Table3).  There  was  a  significant  correlation  between  subjective  stress  and  during  2  session 
Schonstedt  with  no  instrument  condition. 
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Correlations  were  computed  with  PD  and  BAR  between  amylase  and  cortisol  (Table  4). 
There  were  two  significant  negative  correlations  between  salivary  cortisol  for  the  Schonstedt 
with  no  instrument  condition  (D1  PD  and  D3  BAR). 

There  are  a  few  significant  correlations  between  perfonnance  and  stress  measures,  but 
these  are  spurious  and  not  always  in  the  direction  expected.  Over  all  there  is  not  a  strong 
relationship  between  stress  and  perfonnance  in  this  analysis.  This  is  probably  because  the 
operator  stress  levels  are  low  to  moderate.  Operators  are  not  experiencing  high  levels  of  stress 
that  would  impact  perfonnance. 

Comparative  Stress  Data 

The  stress  battery  used  by  the  U.  S.  Army  Research  Laboratory  has  been  tested  and 
validated  in  numerous  studies.  In  Appendix  C  mean  scores  and  the  standard  error  of  the  mean 
(SEM)  for  the  state  MAACL-R  subscale  profiles  after  UXO  detection  are  compared  with  profiles 
obtained  in  relevant  military  studies.  All  comparative  measures  are  stress  perception  measures 
taken  on  the  day  of  the  stressor  following  the  stress  event  except  for  the  independent  control 
(INDEP  CNTRL).  Independent  control  data  were  collected  from  seventeen  non-stressed 
individuals  at  Northwestern  University.  Weapons  competition  (WPN  COMP)  data  were 
collected  from  twenty  infantrymen  engaged  in  a  competitive  weapons  event  (Torre,  Wansack, 
Hudgens,  King,  Fatkin,  Mazurczak,  &  Breitenbach,  1991).  Chemical  Defense  Test  Center 
(CDTF)  data  were  collected  following  exposure  to  active  chemical  agents  at  that  facility  (Fatkin 
&  Hudgens,  1994).  Sustained  operations  data  were  collected  from  twenty-four  soldiers 
following  48  hours  of  sleep  deprivation  (Fatkin,  Knapik,  Patton,  Mullins,  Treadwell,  &  Swann, 
1997).  Fire  Fighting  data  were  collected  from  soldiers  fighting  the  fires  at  Yellowstone  National 
Park  in  1989  (Fatkin,  King,  &  Hudgens,  1990).  Recruiter  data  were  collected  from  287  Army 
recruiters  in  urban,  suburban,  and  rural  locations  around  the  country  (Mullins  &  Fatkin,  2000). 
Data  collected  from  40  UXO  clearance  personnel  working  on  the  Kaho'olawe  Island  UXO 
Clearance  Project  (KAHO)  represent  experienced  UXO  workers  (Morgan  &  Mullins,  2002). 
Patient  decontamination  (patient  decon)  data  represent  Soldiers  performing  patient  litter 
decontamination,  participants  wore  Mission  Oriented  Protective  Posture  (MOPP4)  (Blewett, 
Redmond,  Fatkin,  Popp,  &  Rice,  1995).  The  last  four  bars  of  the  comparative  charts  represent 
data  collected  for  this  project:  experts  wearing  the  Schonstedt  and  TMS  instrumentation  (exp 
Schon),  experts  with  Schonstedt  only  (exp  no  ins),  novices  with  Schonstedt  and  TMS 
instrumentation  (novice  Schon),  and  novices  with  Schonstedt  and  no  TMS  instrumentation 
(novice  no  ins).  With  the  exception  of  novices  without  the  TMS  instrumentation  for  MAACL-R 
hostility  the  means  for  all  UXO  detection  data  was  lower  then  the  independent  control.  Overall 
participants  experience  low  to  moderately  low  levels  of  psychological  stress.  This  corresponds 
with  data  collected  at  the  Kaho'olawe  Island  UXO  Clearance  Project.  This  data  combined  with 
the  Kaho’olawe  findings  indicate  trained  individuals  generally  do  not  experience  high  levels  of 
stress  during  UXO  detection  tasks. 

NASA  TLX 

The  NASA  TLX  is  a  subjective  measure  of  workload.  The  overall  measure  of  workload  is 
based  on  the  weighted  average  of  ratings  on  the  six  subscales  of  mental  demand,  physical 
demand,  temporal  demand,  performance,  effort,  and  frustration.  Means  and  standard  deviations 
for  total  workload  and  the  six  subscales  are  presented  in  Table  2. 
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Novices  reported  a  higher  level  of  overall  workload  while  wearing  the  TMS  instrumentation 
(Figure  1)  and  without  the  TMS  instrumentation  (Figure  2).  Novices  tended  to  give  higher 
ratings  for  the  subscales  mental  demand  and  effort. 

Conclusions 

Overall  stress  perception  data  and  the  physiological  data  indicate  that  these  groups  of 
UXO  clearance  workers  did  not  experience  high  levels  of  stress.  Novices  did  have  introductory 
level  training  in  UXO  detection.  Novices  generally  experienced  slightly  higher  stress  levels  then 
experts.  The  low  stress  levels  experienced  by  test  participants  are  likely  due  to  training. 
Experience  appeared  to  reduce  stress  perceptions  to  near  baseline  levels.  The  low  levels  of  stress 
found  here  are  in  agreement  with  other  UXO  detection  results. 
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Table  1.  Data  Collection  by  UXO  Lanes 


Participant 

During  1 

Schonstedt 

During  2 

During  3 

Post 

No  Instrument 

During  1  During  2 

During  3 

Post 

El 

21-27 

1-9 

10-18 

19-20 

33-25 

24-16 

15-7 

6-1 

E2 

1-9 

10-18 

21-27 

28-33 

33-25 

24-16 

15-7 

6-1 

E3 

1-9 

10-18 

21-27 

28-33 

20-12 

11-3 

33-27 

26-21 

E4 

1-9 

10-18 

21-27 

28-33 

33-25 

24-16 

15-7 

6-1 

E5 

1-9 

10-18 

21-27 

28-33 

33-25 

24-16 

15-7 

6-1 

SI 

1-7 

8-25 

26-33 

33-25 

24-16 

15-7 

6-1 

S2 

21-29 

1-9 

10-18 

19-20 

33-25 

24-16 

15-7 

6-1 

S3 

1-9 

10-18 

21-29 

30-33 

33-25 

24-16 

15-7 

6-1 

S4 

21-29 

30-33 

10-18 

1-9 

33-25 

20-16 

15-7 

6-1 

S5 

21-29 

1-9 

10-18 

19-20 

33-25 

24-16 

15-7 

6-1 
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Table  2.  Stress  measure  correlations  with  corresponding  UXO  PD  Detection  Performance 


ShDl 

Sh  D2 

Sh  D3 

Sh  Post 

NIDI 

NI  D2 

NI  D3 

NI  Post 

MAACL 

Anxiety 

.27 

.17 

.14 

.01 

-.34 

.07 

.14 

.19 

Depression 

-.08 

-.02 

-.15 

-.24 

-.04 

-.30 

-.11 

.20 

Hostility 

.03 

.16 

.13 

.05 

-.05 

-.32 

.13 

-.10 

Dysphoria 

.17 

.18 

.10 

.01 

-.11 

-.02 

.16 

.14 

Positive  Affect 

.22 

.24 

.48 

.19 

.46 

.38 

.10 

.67* 

Subjective  Stress 

-.11 

.17 

.10 

.13 

-.04 

.14 

.40 

-.12 

Subjective  Rating 

.14 

.38 

-.04 

.11 

-.07 

-.12 

-.006 

-.42 

of  Events 

*;  p<=0.05 
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Table  3.  Stress  measure  correlations  with  corresponding  UXO  BAR  Detection  Perfonnance 


ShDl 

Sh  D2 

Sh  D3 

Sh  Post 

NIDI 

NI  D2 

NI  D3 

NI  Post 

MAACL 

Anxiety 

-.21 

-.10 

.40 

.00 

.37 

.05 

.07 

.26 

Depression 

-.41 

-.39 

-.39 

.01 

.58 

.15 

.39 

.25 

Hostility 

-.33 

-.27 

.40 

.05 

.60 

.18 

.12 

.04 

Dysphoria 

-.35 

-.16 

.29 

.04 

.56 

.10 

.13 

.23 

Positive  Affect 

.30 

.12 

.35 

-.14 

.05 

-.12 

-.3 

.19 

Subjective  Stress 

-.26 

-.35 

.37 

-.06 

.54 

.67* 

.06 

-.12 

Subjective  Rating 

-.05 

.09 

.35 

-.27 

.55 

.49 

-.11 

-.42 

of  Events 

*;  p<=0.05 
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Table  4.  UXO  Detection 


AMYD1 

AMY  D2 

AMY  D3 

PD  Schon  with  TMS  -.34 

-.13 

-.02 

BAR  Schon  with  TMS-.  1 1 

.07 

.52 

PD  No  inst.  .38 

.24 

-.28 

BAR  No  inst.  .07 

.16 

.03 

*;  £<=0.05 


with  Salivary  Amylase  and  Cortisol 


AMY  Post 

CORTD1 

CORT  D2 

CORT  D3 

CORT  Post 

-.13 

.52 

.14 

-.18 

.23 

-.22 

-.42 

.20 

-.53 

-.06 

.30 

-.73* 

.12 

.06 

.18 

.19 

-.01 

-.14 

-.75* 

-.41 
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Table  5.  NASA  TLX  Means  and  Standard  Deviations  * 


Schonstedt 

During  1 

During  2 

During  3 

Post 

No  Instrument 

During  1  During  2 

During  3 

Post 

Experts 

Total 

17.5  (19.1) 

27.4(19.0) 

35.6(3.9) 

38.7  (7.4) 

32.5  (17.0) 

35.3  (9.4) 

33.7  (9.2) 

36.8  (7.7) 

Mental 

1.2  (1.2) 

1.1  (1.4) 

2.1  (2.2) 

2.3  (2.2) 

1.8  (1.4) 

3.2  (3.0) 

1.2  (0.6) 

2.1  (1.5) 

Physical 

1.3  (1.7) 

1.4  (1.4) 

1.9  (1.4) 

1.5  (1.3) 

2.1  (2.2) 

1.4  (0.9) 

1.9  (1.6) 

2.7  (2.1) 

Temporal 

1.4  (1.8) 

1.2  (2.1) 

2.5  (2.9) 

1.7  (2.3) 

2.0  (1.8) 

1.8  (1.8) 

2.4  (2.2) 

1.5  (1.1) 

Perfonnance 

10.3  (13.4) 

19.7(13.5) 

22.9(12.2) 

27.1 (6.0) 

20.3  (12.1) 

25.1 (8.9) 

24.3  (8.6) 

25.3  (10.4) 

Effort 

3.3  (5.0) 

4.1  (6.1) 

6.3  (8.5) 

6.0  (7.1) 

6.3  (7.2) 

3.7  (4.8) 

3.9  (4.4) 

5.1  (5.4) 

Frustration 

0.0  (0.0) 

0.0  (0.0) 

0.0  (0.0) 

0.0  (0.0) 

0.0  (0.0) 

0.0  (0.0) 

0.0  (0.0) 

0.0  (0.0) 

Novices 

Total  62.4(20.6)  34.8  (41.4)  44.6(38.3)  37.5  (40.4)  59.1  (15.5)  50.8  (16.5)  60.3  (21.1)  42.4  (33.3) 
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Mental 

15.5  (12.6) 

5.6(10.3) 

10.4(14.0) 

Physical 

5.4  (4.5) 

2.3  (3.1) 

4.0  (5.1) 

Temporal 

5.9  (1.8) 

3.6  (4.7) 

3.5  (2.1) 

Perfonnance 

17.3  (7.8) 

10.5  (14.0) 

10.9(13.7) 

Effort 

17.8(7.1) 

12.8(16.0) 

15.9(14.1) 

Frustration 

0.5  (1.2) 

0.0  (0.0) 

0.0  (0.0) 

*  Standard  deviation  in  parentheses 


8.7(12.9) 

13.9(11.1) 

12.2  (10.4) 

12.3  (12.7) 

5.7  (9.1) 

2.4  (3.1) 

2.3  (2.8) 

3.5  (2.0) 

5.4  (3.1) 

6.7  (6.0) 

2.4  (4.7) 

9.1  (5.0) 

3.3  (4.0) 

5.3  (2.7) 

4.3  (3.3) 

11.9(10.9) 

20.2  (5.7) 

18.6  (4.7) 

20.1  (10.6) 

14.7(13.0) 

12.1 (14.0) 

12.5  (5.7) 

13.3  (6.6) 

15.9(5.3) 

10.5  (9.8) 

0.0  (0.0) 

1.1  (2.4) 

0.0  (0.0) 

1.3  (3.0) 

.53  (1.2) 

NASA  TLX  Total  Workload  with  TMS  Instrument 
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Figure  1.  Mean  total  workload  for  experts  and  novices  using  Schonstedt 

with  TMS  instrumentation. 
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Figure  2.  Mean  total  workload  for  experts  and  novices  using  Schonstedt  without  TMS 

instrumentation. 
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APPENDIX  A 


SAMPLE  DATA  COLLECTION  QUESTIONNAIRES 
General  Information  Questionnaire 
Subjective  Stress 
Subjective  Rating  of  Events 
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Please  answer  all  questions  by  filling  in  the  blanks  as  completely  as  possible.  All 
information  will  be  kept  strictly  confidential. 


ID  # 


BACKGROUND 


1.  Age: 

2.  Ethnicity  _ 

(check  one)  _ 

African  American 
Native  American 

3.  Gender:  Female 

Male 

Caucasian 

(check  one) 

4.  Height 

Weight 

— 

Hispanic 

Asian 

Pacific  Islander 
Other 

5.  Are  you:  Married 

_ Single 

Divorced  (or  in  the  process) 

Widow 

6.  Are  you  a  smoker? 

(circle  one)  Yes 

No 

If  yes, 

How  many  packs  a 

dav?  0-1 

1-2 

2-3 

More  than  3 

7.  Education  completed: 

High  School 

GED 

College 

Grad  School 

(years) 

(yes  or  no) 

(years) 

(years) 

M-21 


8.  Do  you  have  EOD  experience?  (circle  one)  Yes  No 

If  yes,  how  many  years  experience _ 

9.  Do  you  have  combat  engineer  experience?  (circle  one)  Yes  No 

If  yes,  how  many  years  experience _ 

10.  Do  you  have  field  experience  in  UXO  detection?  (circle  one)  Yes  No 

If  yes,  how  many  years  experience _ 

11.  Do  you  have  experience  with  the  Schonstedt  magnetic  locator?  (circle  one)  Yes 

No 

If  yes,  how  many  years  experience _ 

12.  Do  you  have  experience  with  the  EM61  hand  held  system?  (circle  one)  Yes 

No 

If  yes,  how  many  years  experience _ 

13.  Have  you  had  any  prior  military  service?  (circle  one)  Yes  No 

If  yes: 

What  service? _ 

MOS  Primary  _  Time  in  MOS  _  _ 

(years)  (months) 

MOS  Secondary  _  Time  in  MOS  _  _ 

(years)  (months) 

CURRENT  STATUS 

14.  Current  Job  Title _ 

15.  Time  in  Current  Position _  _ 

(years)  (months) 

Brief  Job  Description  _ 
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16.  Present  overall  health:  (check  one) 


(1) 

excellent 

(2) 

good 

(3) 

fair 

(4) 

poor 

17.  How  many  hours  of  sleep  do  you  normally  get  on  week  nights? 

on  weekends?  _ 

18.  Do  you  find  you  are  overtired:  (check  one) 


(1) 

never 

(2) 

occasionally 

(3) 

frequently 

Please  feel  free  to  add  any  other  comments  you  feel  are  important  or  that  we  may  have 
left  out. 


Thank  You  For  Your  Participation! 
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SUBJECTIVE  SCALE 


Circle  ONE  word  that  best  describes  how  you  felt  since  you 
last  completed  this  questionnaire. 

Wonderful 

Fine 

Comfortable 

Steady 

Not  Bothered 
Indifferent 
Timid 
Unsteady 
Nervous 
Worried 
Unsafe 
Frightened 
Terrible 
In  Agony 
Scared  Stiff 
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RATING  OF  EVENTS  -  SPECIFIC 


1.  The  scale  below  represents  a  range  of  how  stressful  an  event 
might  be.  Put  a  check  mark  touching  the  line  (4)  to  rate  how 
much  stress  you  were  experiencing  while  you  were  scanning  for 
unexploded  ordnance  during  this  session. 


Not  at  All  Most  Stress 

Stressful  Possible 


0  10  20  30  40  50  60  70  80  90  100 


2.  At  what  number  value  does  the  check  mark  touch  the  line? 
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APPENDIX  B 


Psychological  and  Psychological 
Stress  Assessment  Graphs 
Comparison  of  Experts  vs.  Novices 

In  Power  Point  file  Report  Appendix  B  Graphs 
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APPENDIX  C 


MAACL-R  Comparison  Graphs 
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APPENDIX  N.  HEARING  TEST  PROCEDURES  AND  RESULTS 
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MEMORANDUM 


TO:  Chris  Appelt,  Aberdeen  Test  Center 

FROM:  Paula  Henry,  Ph.D.,  Research  Audiologist  and  Timothy  Mermagen,  Electrical 
Engineer,  US  Army  Research  Laboratory,  AMSRD-ARL-HR-SD,  Building  520, 
Aberdeen  Proving  Ground,  MD  21005-5425 

RE:  Hearing  tests  performed  on  study  participants,  predicted  effects  of  hearing  loss  on  mine 

detection  performance,  and  comparison  on  mine  detection  devices 


Hearing  test  methods: 

A  certified  audiologist  performed  audiologic  testing  on  all  participants.  Testing  began 
with  an  otoscopic  examination  to  determine  the  status  of  the  pinnae  and  external  auditory  canals. 
Participants  then  completed  pure  tone  air  conduction  testing.  The  testing  took  place  in  a  sound 
treated  booth  through  the  use  of  a  clinical  diagnostic  audiometer  (Interacoustics  AC40).  Pure 
tones  at  octave  frequencies  250  -  8000  Hz  and  interoctaves  3000  and  6000  Hz  were  presented 
through  TDH-39  superaural  headphones.  Middle  ear  status  was  determined  through 
tympanometry  using  a  Grason-Stadler  37  Auto  Tympanometer. 

Hearing  test  results: 

Results  of  audiologic  testing,  as  shown  in  Figure  1,  revealed  that  on  average  the 
participants  had  normal  hearing  sensitivity  (defined  as  air  conduction  thresholds  of  20  dB  HL  or 
better)  in  both  ears  and  all  participants  showed  nonnal  bilateral  middle  ear  function.  After 
examination  of  individual  hearing  tests,  one  participant,  El,  was  found  to  have  mild-to-moderate 
high  frequency  hearing  loss  (3000-8000  Hz)  in  both  ears.  Two  additional  participants,  S2  and  S4, 
had  moderate  hearing  loss  in  the  high  frequencies,  but  only  in  one  ear.  For  participant  S2,  the 
hearing  loss  was  in  the  left  ear  and  for  S4,  the  loss  was  is  the  right  ear. 


Average  Thresholds  for  All  Participants 


Frequency  (Hz) 


100  1000  10000 


X —  Left  Ear 


Right  Ear 


Figure  1.  Average  hearing  thresholds  for  all  participants.  Error  bars 
indicate  +/-  1  standard  deviation. 
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Measurements  from  magnetic  locator  devices: 

Recordings  from  the  two  Schonstadt  magnetic  locators  used  in  the  study,  were  made  in 
the  following  manner.  A  Knowles  Electronics  Manikin  for  Acoustic  Research  (KEMAR)  was 
used  with  Etymotic  E-ll  microphones  mounted  in  the  ear  canals.  The  acoustic  output  from  the 
Schonstadt  was  routed  through  a  Bogen  Communications,  Inc.  10  Watt  attenuator  to  the  Audio- 
Technica  ATH  M-30  headphones  which  were  placed  over  the  manikin’s  ears.  The  particular 
Schonstadt  device  that  the  participant  used  was  held  in  their  preferred  calibration  position,  set  to 
their  selected  instrument  sensitivity  and  volume  control  setting.  The  output  of  the  headphones 
was  recorded  through  the  E-ll  microphones  into  a  Vaio  notebook  computer  using  a  OldB  sound 
analysis  system.The  frequency  responses  of  the  stimuli  were  measured  in  1/3  octave  bands.  The 
hearing  thresholds  measured  on  the  participants  were  converted  from  dB  HL  (as  measured  for 
the  hearing  tests)  to  dB  SPL  in  order  to  equate  the  signals.  Figures  2  through  13  show  the 
frequency  responses  of  the  acoustic  stimuli  recorded  from  the  Schonstadt  magnetic  locator  along 
with  the  participant’s  hearing  thresholds  for  comparison.  The  difference  between  the  stimuli 
response  and  the  participants’  thresholds  is  the  amount  of  the  stimulus  that  was  audible  to  the 
user. 

For  all  participants,  hearing  status  was  not  expected  to  have  a  significant  impact  on 
performance  in  the  study  for  three  reasons.  First,  the  signals  from  the  magnetic  locators 
presented  to  the  listeners  were  provided  through  headphones  to  both  ears.  Therefore,  hearing  loss 
in  only  one  ear  (participants  S2  and  S4)  should  not  impact  overall  performance  as  the  better  ear 
would  be  able  to  compensate  for  the  loss  in  the  poorer  ear.  Second,  the  signals  provided  to  the 
listeners  were  of  a  level  high  enough  to  be  above  their  hearing  thresholds  as  shown  in  the  graphs. 
Third,  the  signals  emitted  from  the  magnetic  locators  were  broad  in  their  frequency  spectrum,  so 
that  an  individual  with  a  bilateral  high  frequency  hearing  loss  (El)  should  have  been  able  to 
utilize  the  lower  frequency  infonnation  in  the  signal.  Indeed,  in  the  case  of  participant  El,  the 
frequency  range  of  the  signal  from  500  to  3000  Hz  was  between  10  and  70  dB  above  threshold. 
Comparisons  of  Schonstadt  devices: 

Concern  was  raised  as  to  the  equality  of  the  stimuli  provided  by  the  two  Schonstadt 
devices.  In  order  to  compare  the  two,  recordings  were  made  of  the  signal  emitted  from  each  of 
the  devices  under  the  same  conditions.  A  shot,  used  in  shot  put,  was  placed  under  a  bucket  of  a 
particular  height.  The,  the  sensitivity  control  and  the  volume  control  on  the  two  devices  were  set 
to  the  same  levels.  Recordings  of  the  signal  from  the  two  Schonstadts  are  shown  in  Figure  14.  As 
can  be  seen  in  the  figure,  very  little  difference  is  seen  between  the  stimuli  recorded  from  the  two 
instruments.  Based  on  this  figure,  it  is  reasonable  to  state  that  the  two  Schonstadts  are 
sufficiently  equivalent. 
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Figure  7.  Same  as  Figure  2,  for  participant  El.  Note  that  participant  has  hearing  loss  above 
3000  Hz,  but  would  have  audibility  of  the  signal  between  500  and  3000  Hz. 


Figure  9.  Same  as  Figure  2,  for  participant  E3. 
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Figure  10.  Same  as  Figure  2,  for  participant  E4. 
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Figure  1 1 .  Same  as  Figure  2,  for  participant  E5. 
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Figure  12.  Same  as  Figure  2,  for  participant  Rl. 
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Figure  13.  Same  as  Figure  2,  for  participant  R2. 
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Comparison  of  Schonstadt  Devices  -  Signal 
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Figure  14.  Comparison  of  the  signals  emitted  from  the  two  Schonstadt  devices  recorded  under 
the  same  condition.  The  devices  were  held  directly  over  the  bucket  with 
the  shotput  beneath  it  and  the  instrument  sensitivity  set  at  4. 
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CSTE-DTC-AT-AD-R 


19  May  2006 


MEMORANDUM  FOR  CHIEF,  Military  Environmental  Technology  Demonstration  Center 
Team,  ATTN:  Chris  Appelt 

SUBJECT:  Performance  Characteristics  and  Measurements  of  Operators  without  TMS 
Instrumentation 

1.  Background 

a.  Ten  operators  were  chosen  to  conduct  this  test.  Five  were  considered  novices  with 
little  to  no  experience  with  UXO  searching,  while  the  remaining  five  were  considered  experts 
with  twelve  months  or  more  UXO  experience.  Thirty-three  lanes  were  scattered  with  60  UXO, 
consisting  of  40mm,  60mm,  8 limn,  105mm,  and  155mm  rounds.  The  burial  depths  of  these 
items  were  6  inches,  12  inches,  18  inches,  24  inches,  and  30  inches,  respectively.  The  UXO 
were  randomly  placed  throughout  the  lanes  and  were  arbitrarily  orientated  horizontally  or 
vertically.  Table  1  presents  the  number  and  orientation  of  each  ordnance  type  used  for  this  test. 


Table  1.  Summary  of  Ordnance  Type  and  Orientation 


40mm 

60mm 

81mm 

105mm 

155mm 

Total 

H 

V 

H 

V 

H 

V 

H 

V 

H 

V 

4 

5 

5 

7 

6 

7 

5 

8 

6 

7 

60 

9 

12 

13 

13 

13 

b.  Each  operator  used  the  Schonstadt  magnetometer  to  traverse  all  thirty -three  lanes 
twice,  once  with  just  the  Schonstadt  and  again  with  the  Schonstadt  equipped  with  two  sensors 
allowing  the  TMS  system  to  track  and  record  the  coordinates  of  the  sensors  at  a  rate  of  ten  Hertz. 
The  data  in  this  memorandum  are  of  the  operators  using  the  Schonstadt  without  the  TMS 
instrumentation,  except  where  comparisons  are  made  between  with  and  without  instrumentation. 

c.  In  addition  to  the  ten  operators,  two  quality  control  operators  traversed  through  the  33 
lane  grid  without  instrumentation  with  the  EM-61.  Also,  two  rookies  with  no  UXO  experience 
or  training  went  through  the  first  seven  lanes  with  the  Schonstadt  magnetometer. 

2.  Scoring  Methodology 

a.  The  actual  coordinates  of  emplaced  UXO  are  called  ground  truth.  Each  operator’s  data 
was  scored  against  the  ground  truth.  The  findings  made  by  an  operator  are  called  anomalies.  If 
an  anomaly  was  within  a  one-meter  radius  halo  of  the  ground  truth,  it  was  considered  a  hit. 
Anomalies  outside  this  radius  were  considered  false  alanns.  If  there  were  multiple  anomalies 
within  a  halo,  only  one  was  counted  as  a  hit,  while  the  others  were  not  considered  false  alarms. 

b.  Three  performance  measurements  were  calculated  for  each  operator  -  Probability  of 
Detection  (Pd),  False  Alarm  Rate  (FAR),  and  a  numerical  combination  of  Pd  and  FAR,  Distance 
from  Optimal  Point  (DOP).  It  is  assumed  that  the  number  of  correct  hits  is  a  binomially 
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distributed  random  variable.  FAR  was  calculated  by  the  number  of  false  alarms  divided  by  the 
area  traversed  by  the  operator.  On  the  graph  of  Pd  versus  FAR,  the  closer  an  operator  is  to  (0,1), 
(zero  FAR  and  100%  Pd,  or  the  optimal  point),  the  better  the  operator  did.  The  DOP  was 
calculated  as  the  distance  from  (0,1)  to  an  operator’s  point  (FAR,  Pd).  The  smaller  DOP  number, 
the  better  the  operator  did. 

c.  Two  performance  characteristics,  lane  velocity  and  total  time,  were  obtained  for  each 
operator  without  instrumentation.  Lane  velocity  is  defined  by  the  lane  length  divided  by  the  time 
each  operator  took  to  complete  each  lane.  This  performance  characteristic  is  the  average  lane 
velocity  over  all  thirty-three  lanes. 

d.  The  objective  of  this  phase  of  testing  was  to  observe  and  record  UXO  Technicians  while 
executing  a  “mag  and  flag”  operation  on  the  pre-seeded  test  grid. 

3.  Test  Findings 

a.  The  DOP  parameter  is  a  combination  of  Pd  and  FAR  and  is  therefore  an  overall 
performance  measurement.  Figure  1  depicts  the  relationship  between  DOP  and  UXO 
Experience.  The  expected  outcome  of  this  plot  would  be  a  linear  correlation  with  negative  slope 
of  low  experience  receiving  high  DOP  and  high  experience  receiving  low  DOP.  To  assess  the 
linear  correlation,  the  R2  term  is  used.  R2  is  a  descriptive  measure  between  zero  and  one  that 
measures  how  well  the  linear  line  approximates  real  data  points.  The  closer  R  is  to  one,  the 
better  the  sample  is  at  predicting  the  true  population.  From  Figure  1  it  can  be  concluded  that 
there  seems  to  be  no  correlation  between  experience  level  and  optimal  perfonnance. 


Figure  1 .  Distance  from  Optimal  Point  versus  UXO  Experience  of  All  Operators  without 

Instrumentation. 
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b.  Table  2  displays  the  average  Pd  for  each  ordnance  type/depth  and  orientation  for  all  the 
operators  without  instrumentation.  All  operators  achieved  100%  detection  rates  on  the  40mm 
both  when  horizontally  and  vertically  oriented.  Operators  had  the  lowest  detection  rates  on  the 
81mm  both  horizontally  and  vertically  oriented.  Overall,  vertically  oriented  ordnance  had  higher 
Pd  than  horizontally  oriented  ordnance.  Note  also  from  Table  2  that  ordnance  type  and  depth  are 
codependent;  that  is,  since  each  ordnance  type  was  buried  at  a  certain  depth,  no  information  can 
be  deduced  about  ordnance  type  or  depth  separately. 


Table  2.  Average  Pd  of  All  Operators  without  Instrumentation  by  Ordnance  Type/Depth  and 

Orientation. 


Type 

Depth,  in 

Horizontal 

Vertical 

Total 

40mm 

6 

100.0% 

100.0% 

100.0% 

60mm 

12 

94.0% 

98.6% 

96.7% 

81mm 

18 

80.0% 

92.9% 

88.5% 

105mm 

24 

100.0% 

97.5% 

98.5% 

155mm 

30 

100.0% 

98.6% 

99.2% 

Total 

94.2% 

97.9% 

96.3% 

4.  Technical  Assessment 


The  ten  operators  were  compared  in  three  different  groupings:  five  experts  versus  live 
novices,  top  live  performers  versus  bottom  live  performers,  and  three  highest  performers  versus 
four  middle  performers  versus  three  lowest  performers.  The  latter  two  comparisons  are  for 
informational  purposes  and  are  presented  in  Appendix  A.  The  Chi-Square  Test  for  differences  in 
proportions  and  the  Mann- Whitney  test  were  used  to  statistically  compare  the  perfonnance  data 
of  each  of  these  breakdowns.  The  Mann- Whitney  test  is  a  nonparametric  technique  that  uses  the 
ranked  order  of  the  data  to  see  if  two  samples  are  identical  when  nothing  about  the  underlying 
distributions  are  known,  {reference:  Conover  W.  J.  Practical  Nonparametric  Statistics.  2nd 
Edition.  New  York:  John  Wiley  and  Sons,  1980.} 
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a.  Experts  versus  Novices 

1)  Table  3  presents  a  summary  of  the  performance  data  of  each  expert  and  novice  without 
instrumentation.  Figure  2  illustrates  the  Pd  versus  FAR  of  experts  and  novices. 


Table  3.  Summary  of  Performance  Data  of  Experts  and  Novices  without  Instrumentation. 


Operator 

Probability 
of  Detection 
(Pd) 

False 

Alarm 

Rate 

(1/mA2) 

DOP 

Lane 

Velocity 

(m/s) 

Total 
Time  (s) 

E-1 

100.0% 

0.154 

0.154 

0.095 

8238 

E-2 

91.7% 

0.072 

0.110 

0.088 

9086 

</> 

t 

E-3 

100.0% 

0.087 

0.087 

0.100 

8279 

0 

Q. 

E-4 

93.3% 

0.034 

0.075 

0.092 

8608 

X 

LU 

E-5 

93.3% 

0.090 

0.112 

0.096 

8199 

Mean 

95.7% 

0.087 

0.108 

0.094 

8482 

Std.  Dev. 

0.0401 

0.0435 

0.0302 

0.0044 

374.7 

N-1 

100.0% 

0.057 

0.057 

0.104 

7612 

N-2 

98.3% 

0.028 

0.033 

0.112 

6810 

</> 

0 

N-3 

95.0% 

0.047 

0.069 

0.124 

6661 

> 

N-4 

98.3% 

0.168 

0.169 

0.050 

17073 

o 

z 

N-5 

93.3% 

0.017 

0.069 

0.094 

8733 

Mean 

97.0% 

0.063 

0.079 

0.097 

9378 

Std.  Dev. 

0.0274 

0.0605 

0.0522 

0.0284 

4379.6 

Figure  2.  Pd  versus  FAR  of  Experts  and  Novices  without  Instrumentation. 
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2)  Using  the  Chi-Square  Distribution  at  0.05  significance  level,  Pd  between  experts  and 
novices  without  instrumentation  was  found  not  to  be  significantly  different.  Using  the  Mann- 
Whitney  Test  at  the  0.05  significance  level,  no  significant  differences  were  found  between  the 
number  of  false  alarms,  DOP,  and  time  between  the  novices  and  experts. 

3)  Table  4  presents  the  Pd  by  ordnance  orientation  and  type  for  both  novices  and  experts. 
Novices  had  100%  Pd  both  horizontally  and  vertically  with  three  different  types  of  ordnance: 
40mm,  105mm,  and  155mm.  They  also  did  equal  to  or  better  than  the  Experts  in  every  category 
except  two,  the  60mm  horizontally  and  vertically.  Overall,  the  Novices  averaged  1.1%  higher 
Pd’s  than  the  Experts. 


Table  4.  Experts  vs.  Novices  without  Instrumentation  -  Pd  by  Ordnance  Orientation  and  Type. 


n 

Type 

Novices 

Experts 

Total 

Pd  Differences 

Horizontal 

40mm 

100.0% 

100.0% 

100.0% 

0.0% 

60mm 

92.0% 

96.0% 

94.0% 

-4.0% 

81mm 

83.3% 

76.7% 

80.0% 

6.7% 

105mm 

100.0% 

100.0% 

100.0% 

0.0% 

155mm 

100.0% 

100.0% 

100.0% 

0.0% 

Total 

94.6% 

93.8% 

94.2% 

Vertical 

40mm 

100.0% 

100.0% 

100.0% 

0.0% 

60mm 

97.1% 

100.0% 

98.6% 

-2.9% 

81mm 

94.3% 

91.4% 

92.9% 

2.9% 

105mm 

100.0% 

95.0% 

97.5% 

5.0% 

155mm 

100.0% 

97.1% 

98.6% 

2.9% 

Total 

98.8% 

97.1% 

97.9% 

Overall  Total  97.0%  95.7%  96.3% 

4)  Figures  3-5  show  the  comparison  of  average  lane  velocity  to  the  three  performance 
measurements,  Pd,  FAR,  and  DOP  respectively.  Two  trends  can  be  seen  from  these  plots  as  the 
quicker  the  novices  traverse  the  lanes:  the  fewer  false  alarms  they  indicate  and  they  also  have  a 
better  overall  perfonnance  (low  DOP).  Also,  the  experts  had  little  variation  among  themselves 
for  lane  velocity  (standard  deviation  equal  to  .0044).  The  novices,  on  the  other  hand,  did  have 
high  variation  for  lane  velocity  (standard  deviation  equal  to  .0284). 

5)  Figures  6-8  show  the  comparison  of  total  time  to  the  three  performance  measurements. 
Consistent  with  average  lane  velocity,  the  experts  had  less  variation  among  themselves  then  the 
novices  for  total  time.  Also,  the  more  time  the  novices  took,  the  more  false  alanns  they 
indicated. 
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Figure  3.  Experts  and  Novices  -  Pd  vs.  Average  Lane  Velocity. 
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Figure  5.  Experts  and  Novices  -  DOP  vs.  Average  Lane  Velocity. 
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Figure  4.  Experts  and  Novices  -  FAR  vs.  Average  Lane  Velocity 
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Figure  6.  Experts  and  Novices  -  Pd  vs.  Total  Time. 
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Figure  8.  Experts  and  Novices  -  DOP  vs.  Total  Time. 
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Figure  7.  Experts  and  Novices  -  FAR  vs.  Total  Time. 


b.  Operators  with  Instrumentation 

1)  For  comparison  purposes,  a  summary  of  the  performance  data  of  the  experts  and  novices 
with  instrumentation  for  all  33  lanes  is  presented  in  Table  5.  Figure  9  illustrates  the  Pd  versus 
FAR  of  experts  and  novices. 


Table  5.  Summary  of  Performance  Data  of  Experts  vs.  Novices  with  Instrumentation. 


Operator 

Probability 
of  Detection 
(Pd) 

False 

Alarm 

Rate 

(1/mA2) 

DOP 

Lane 

Velocity 

(m/s) 

Total 
Time  (s) 

E-1 

98.3% 

0.048 

0.051 

0.089 

9304 

E-2 

83.3% 

0.061 

0.177 

0.075 

10602 

</> 

tr 

E-3 

100.0% 

0.087 

0.087 

0.074 

10661 

<D 

Q. 

E-4 

61.7% 

0.012 

0.384 

0.105 

7574 

X 

in 

E-5 

95.0% 

0.096 

0.109 

0.069 

11626 

Mean 

87.7% 

0.061 

0.161 

0.082 

9953 

Std.  Dev. 

0.1593 

0.0333 

0.1325 

0.0147 

1565.4 

N-1 

98.3% 

0.020 

0.026 

0.088 

8848 

N-2 

98.3% 

0.013 

0.021 

0.110 

7074 

</) 

<D 

N-3 

95.0% 

0.016 

0.052 

0.097 

9097 

> 

N-4 

100.0% 

0.042 

0.042 

0.035 

23347 

o 

z 

N-5 

100.0% 

0.015 

0.015 

0.106 

7562 

Mean 

98.3% 

0.021 

0.032 

0.087 

11186  j 

Std.  Dev. 

0.0204 

0.0120 

0.0155 

0.0302 

6851.3 

Figure  5.  Pd  versus  FAR  of  Experts  and  Novices  with  Instrumentation 
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2)  Using  the  Chi-Square  Distribution  at  0.05  significance  level,  the  novice  Pd  with 
instrumentation  was  found  to  be  significantly  greater  than  the  expert  Pd  with  instrumentation. 
Using  the  Mann-Whitney  Test  at  the  0.05  significance  level,  no  significant  differences  were 
found  between  the  number  of  false  alarms  and  time,  but  the  DOP  of  the  novices  was  significantly 
less  than  the  experts. 

c.  Quality  Control  Operators 

Two  quality  control  operators  (W-l  and  W-2)  went  through  the  33  lane  grid  without 
instrumentation  with  the  EM-61.  They  performed  slightly  below  the  overall  Pd  average  and 
around  the  same  as  the  FAR  average.  Table  6  is  a  summary  of  their  performance,  and  Figures  10 
and  1 1  are  plots  of  all  operators  Pd  and  FAR. 


Table  6.  Summary  of  Quality  Control  Operators  without  Instrumentation  for  Lanes  1-33. 


Operator 

#Targets 

#1-1  its 

#Misses 

Probability  of 
detection  (Pd) 

#  False 

Alarms 

FA  Rate, 
cnts/rn^ 

#Mult 

Hits 

W-1 

60 

57 

3 

0.950 

83 

0.073 

14 

W-2 

60 

55 

5 

0.917 

72 

0.064 

14 
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Figure  10.  Probability  of  Detection  of  All  Operators  Without  Instrumentation  for  Lanes  1-33. 
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Figure  11.  False  Alarm  Rate  of  All  Operators  Without  Instrumentation  for  Lanes  1-33. 
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d.  Lanes  1-20 


2 

Lanes  1  -  20  were  situated  north  to  south,  and  were  23.703  meters  by  1.5  meters  (35.6  m  ). 
There  were  28  ordnance  scattered  throughout  these  twenty  lanes.  Tables  7  and  8  present  the 
performance  data  of  all  the  operators  with  and  without  instrumentation  of  only  lanes  1-20. 


Table  7.  Summary  Table  of  All  Operators  with  Instrumentation  for  Lanes  1-20. 


1 

Operator 

Probability 
of  Detection 
(Pd) 

m 

DOP 

Lane 

Velocity 

(m/s) 

if) 

+j 

CD 

Q_ 

E-1 

100.0% 

0.052 

0.052 

0.101 

4904 

E-2 

82.1% 

0.082 

0.196 

0.068 

7259 

E-3 

100.0% 

0.107 

0.107 

0.070 

7030  i 

E-4 

60.7% 

0.020 

0.393 

0.094 

5247 

X 

LU 

E-5 

92.9% 

0.136 

0.154 

0.062 

7907 

Mean 

87.1% 

0.079 

0.181 

0.079 

6469 

Std.  Dev. 

0.1648 

0.0456 

0.1306 

0.0172 

1318.1 

Novices 

N-1 

96.4% 

0.024 

0.043 

0.084 

5806 

N-2 

96.4% 

0.014 

0.038 

0.120 

4049  | 

N-3 

96.4% 

0.018 

0.040 

0.087 

6489 

N-4 

100.0% 

0.051 

0.051 

0.038 

13696 

N-5 

100.0% 

0.017 

0.017 

0.118 

4243  j 

Mean 

97.9% 

0.025 

0.038 

0.090 

6857 

Std.  Dev. 

0.0196 

0.0149 

0.0126 

0.0331 

3960.1 

Table  8.  Summary  Table  of  All  Operators  without  Instrumentation  for  Lanes  1-20. 
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Operator 

Probability 
of  Detection 
(Pd) 

False 

Alarm 

Rate 

11/mA21 

DOP 

Lane 

Velocity 

(m/s) 

Total 
Time  (s) 

< n 

j— 

CD 

Q_ 

E-1 

100.0% 

0.198 

0.198 

0.091 

5371 

E-2 

85.7% 

0.093 

0.170 

0.095 

5304 

E-3 

100.0% 

0.124 

0.124 

0.092 

5634 

E-4 

89.3% 

0.048 

0.117 

0.095 

5223 

X 

LU 

E-5 

96.4% 

0.120 

0.125 

0.091 

5419 

Mean 

94.3% 

0.116 

0.147 

0.093 

5390 

Std.  Dev. 

0.0649 

0.0548 

0.0357 

0.0020 

154.9 

Novices 

N-1 

100.0% 

0.083 

0.083 

0.101 

4916 

N-2 

100.0% 

0.042 

0.042 

0.109 

4400 

N-3 

96.4% 

0.055 

0.065 

0.141 

3685 

N-4 

100.0% 

0.167 

0.167 

0.060 

8498 

N-5 

92.9% 

0.023 

0.075 

0.112 

4475 

Mean 

97.9% 

0.074 

0.087 

0.105 

5195 

Std.  Dev. 

0.0319 

0.0566 

0.0477 

0.0292 

1898.6 

o 

a 

W-1 

92.9% 

0.097 

0.120 

W-2 

100.0% 

0.089 

0.089 

Mean 

96.4% 

0.093 

0.105 

e.  Lanes  21-33 

Lanes  21-33  were  positioned  east  to  west.  Lanes  21-26  were  17.55  meters  by  1.5  meters 
(26.325  in2),  and  lanes  27  -  33  were  25.00  meters  by  1.5  meters  (37.5  m2).  There  were  32 
ordnance  placed  throughout  these  thirteen  lanes.  Tables  9  and  10  present  the  performance  data 
of  all  the  operators  with  and  without  instrumentation  of  only  lanes  21-33. 
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Table  9.  Summary  Table  of  All  Operators  with  Instrumentation  for  Lanes  21-33. 


1 

Operator 

Probability 
of  Detection 
(Pd) 

■ 

DOP 

Lane 

Velocity 

(m/s) 

E-1 

96.9% 

0.040 

0.051 

0.070 

4400 

E-2 

84.4% 

0.026 

0.158 

0.085 

3343 

C I ) 

•4— < 
L_ 

E-3 

100.0% 

0.052 

0.052 

0.081 

3631 

CD 

Q- 

E-4 

62.5% 

0.000 

0.375 

0.121 

2327 

X 

Hi 

E-5 

96.9% 

0.029 

0.042 

0.079 

3719 

Mean 

88.1% 

0.029 

0.136 

0.087 

3484 

Std.  Dev. 

0.1553 

0.0195 

0.1420 

0.0198 

754.1 

N-1 

100.0% 

0.014 

0.014 

0.093 

3042 

N-2 

100.0% 

0.012 

0.012 

0.094 

3025 

(/) 

<D 

N-3 

93.8% 

0.012 

0.064 

0.112 

2608 

O 

> 

N-4 

100.0% 

0.029 

0.029 

0.030 

9651 

o 

z 

N-5 

100.0% 

0.012 

0.012 

0.087 

3319 

Mean 

98.8% 

0.016 

0.026 

0.083 

4329 

Std.  Dev. 

0.0280 

0.0073 

0.0221 

0.0310 

2985.9 

Table  10.  Summary  Table  of  All  Operators  without  Instrumentation  for  Lanes  21-33. 


1 

Operator 

Probability 
of  Detection 
(Pd) 

False 

Alarm 

Rate 

(1/mA2) 

DOP 

Lane 

Velocity 

(m/s) 

Total 
Time  (s) 

Experts 

E-1 

100.0% 

0.078 

0.078 

0.101 

2867 

E-2 

96.9% 

0.036 

0.047 

0.078 

3782 

E-3 

100.0% 

0.026 

0.026 

0.111 

2645 

E-4 

96.9% 

0.010 

0.033 

0.086 

3385 

E-5 

90.6% 

0.040 

0.102 

0.103 

2780 

Mean 

96.9% 

0.038 

0.057 

0.096 

3092 

Std.  Dev. 

0.0383 

0.0255 

0.0321 

0.0133 

476.9 

Novices 

N-1 

100.0% 

0.012 

0.012 

0.108 

2696 

N-2 

96.9% 

0.005 

0.032 

0.117 

2410 

N-3 

93.8% 

0.033 

0.071 

0.098 

2976 

N-4 

96.9% 

0.169 

0.172 

0.035 

8575 

N-5 

93.8% 

0.007 

0.063 

0.068 

4258 

Mean 

96.3% 

0.045 

0.070 

0.085 

4183 

Std.  Dev. 

0.0261 

0.0701 

0.0617 

0.0338 

2554.7 

O 

O 

W-1 

96.9% 

0.033 

0.046 

W-2 

84.4% 

0.021 

0.158 

Mean 

90.6% 

0.027 

0.102 

0-15 


f.  Lanes  1-20  vs.  Lanes  21-33 


A  statistical  analysis  was  done  to  compare  perfonnance  data  from  lanes  1-20  (north-south 
lanes)  to  lanes  21-33  (east-west  lanes).  Pd  was  tested  using  the  Chi-Square  Test  at  0.05 
significance  level,  while  the  number  of  false  alarms,  distance  measurement,  and  total  time  were 
tested  using  the  Mann- Whitney  Test  at  the  0.05  significance  level.  Table  11  shows  the 
performance  measurements  that  were  significantly  different  between  experts  and  novices.  With 
instrumentation  for  all  lanes,  the  novices  had  significantly  better  Pd  and  significantly  shorter 
DOP.  Without  instrumentation  for  all  lanes,  the  novices  and  experts  perfonned  similarly.  Table 
12  presents  the  significant  difference  between  lanes  1-20  and  lanes  21-33  data.  (Total  time  was 
not  addressed  in  these  comparisons  because  the  area  covered  was  not  the  same.)  For  experts  and 
novices  both  with  and  without  instrumentation,  there  were  significantly  more  false  alanns  found 
for  lanes  1-20  than  lanes  21-33.  Common  False  Alarms  (CFA)  are  false  alarms  in  which  four  or 
more  operators  detected  a  FA  within  a  .25  meter  radius  circle.  For  the  operators  without 
instrumentation  (78  CFA),  86%  occurred  in  lanes  1-20.  For  the  operators  with  instrumentation 
(38  CFA),  89%  occurred  in  lanes  1-20. 


Table  11.  Summary  of  Significance  Testing  of  Experts  vs.  Novices. 


Pda 

E2Z3B 

DOPb 

with 

Inst 

Lanes  1-33 

SIG 

— 

SIG 

— 

Lanes  1-20 

SIG 

SIG 

SIG 

— 

Lanes  21-33 

SIG 

- 

SIG 

- 

without 

Inst 

Lanes  1-33 

— 

— 

— 

— 

Lanes  1-20 

— 

— 

SIG 

— 

Lanes  21-33 

- 

- 

- 

- 

aChi-Square  Distribution,  one-sided  test,  at  0.05  significance  level 
bMann- Whitney  Test,  one-sided  test,  at  0.05  significance  level 


Table  12.  Summary  of  Significance  Testing  of  Data  from  Lanes  1-20  vs.  Lanes  21-33. 


Pda 

#  FAb 

DOPb 

with 

Inst 

Experts 

— 

SIG 

— 

Novices 

- 

SIG 

- 

without 

Inst 

Experts 

— 

SIG 

SIG 

Novices 

- 

SIG 

- 

aChi-Square  Distribution,  one-sided  test,  at  0.05  significance  level 
hMann- Whitney  Test,  one-sided  test,  at  0.05  significance  level 
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g.  Operator  data  with  Instrumentation  versus  without  Instrumentation 

A  statistical  analysis  was  done  to  compare  performance  data  with  instrumentation  and 
without  instrumentation.  Pd  was  tested  using  the  Chi-Square  Distribution  at  0.05  significance 
level,  while  the  number  of  false  alarms,  distance  measurement,  and  total  time  were  tested  using 
the  Mann-Whitney  Test  at  the  0.05  significance  level.  Table  13  shows  the  performance 
measurements  that  were  significantly  different  between  the  operators  with  and  without 
instrumentation.  For  all  lanes,  the  experts  had  significantly  higher  Pd  without  instrumentation, 
and  the  novices  had  significantly  greater  number  of  false  alarms  and  greater  DOP  without 
instrumentation.  Figures  12-15  graphically  depict  the  difference  between  the  operators  with  vs. 
without  instrumentation  for  probability  of  detection,  false  alarm  rate,  distance  from  optimal 
point,  and  time,  respectively. 


Table  13.  Summary  of  Significance  Testing  of  Operators  with  Instrumentation  and  without 

Instrumentation. 


Pda 

#  FAb 

DOPb 

Total  Timeb 

Lanes 

1-33 

Experts 

SIG 

- 

- 

- 

Novices 

- 

SIG 

SIG 

- 

Lanes 

1-20 

Experts 

SIG 

- 

- 

- 

Novices 

- 

SIG 

SIG 

- 

Lanes 

21-33 

Experts 

SIG 

- 

- 

- 

Novices 

- 

- 

- 

- 

aChi-Square  Distribution,  one-sided  test,  at  0.05  significance  level 
hMann- Whitney  Test,  one-sided  test,  at  0.05  significance  level 
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Figure  12.  Difference  in  Probability  of  Detection  between  Operators  without 
Instrumentation  and  with  Instrumentation. 
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Figure  13.  Difference  in  False  Alarm  Rate  between  Operators  without  Instrumentation 

and  with  Instrumentation. 


Figure  14.  Difference  in  Distance  from  Optimal  Point  between  Operators  without 
Instrumentation  and  with  Instrumentation. 
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Figure  15.  Difference  in  Total  Time  between  Operators  without  Instrumentation  and 

with  Instrumentation. 
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h.  Rookie  Data 


Two  rookies  with  no  UXO  experience  or  training  were  asked  to  traverse  the  first  seven 
lanes  with  the  Schonstadt.  Table  14  presents  the  results  of  rookies  along  with  the  other  ten 
operators  without  instrumentation  and  two  quality  control  operators.  One  rookie,  R-2,  had  as 
good  a  Pd  as  the  expert  average  (.917).  The  other  rookie,  R-l,  had  a  poor  performance,  with  the 
lowest  Pd  of  all  the  operators  and  high  FAR.  Figure  16  is  Pd  versus  FAR  plot  of  all  fourteen 
operators  without  instrumentation  for  only  lanes  1-7. 


Table  14.  Summary  Table  of  Test  Operators,  Quality  Control  Operators,  and  Rookies  without 

Instrumentation  Lanes  1-7. 


Operator 

Rating 

#Targets 

#Hits 

#Misses 

Probability  of 
detection  (Pd) 

#  False 

Alarms 

FA  Rate, 
cnts/nf 

#Mult 

Hits 

E-1 

Expert 

12 

12 

0 

1.000 

31 

0.125 

3 

E-2 

m 

12 

10 

2 

0.833 

18 

0.072 

1 

E-3 

ii 

12 

12 

0 

1.000 

18 

0.072 

1 

E-4 

ii 

12 

10 

2 

0.833 

8 

0.032 

1 

E-5 

ii 

12 

11 

1 

0.917 

19 

0.076 

2 

MEAN 

11.0 

1.0 

0.917 

18.8 

0.076 

1.6 

N-1 

Novice 

12 

12 

0 

1.000 

22 

0.088 

3 

N-2 

ii 

12 

12 

0 

1.000 

8 

0.032 

1 

N-3 

ii 

12 

11 

1 

0.917 

7 

0.028 

3 

N-4 

ii 

12 

12 

0 

1.000 

27 

0.108 

1 

N-5 

ii 

12 

11 

1 

0.917 

4 

0.016 

1 

MEAN 
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Figure  16.  All  Operators  without  Instrumentation  for  Lanes  1-7. 
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i.  Demographic  Data 


1)  Table  15  presents  the  demographic  information  about  the  ten  operators.  For  analysis, 
the  age,  education,  and  years  of  UXO  experience  were  broken  into  three  groups.  Table  16 
displays  these  groups  and  the  operators  that  fall  into  each  group. 


Table  15.  Summary  Table  of  Operator  Demographics 


1 

Operator 

Age 

Gender 

Years  of 

Education 

Months  of 

EOD 

Experience 

Months  of 

UXO 

Experience 

Months  of 

Schonstadt 

Experience 

Prior 

Military 

Experience 

in 

-*-> 

SB 

Q. 

X 

LU 

E-1 

34 

Male 

12 

120 

48 

48 

Yes 

E-2 

37 

Male 

12 

156 

72 

72 

Yes 

E-3 

28 

Male 

13 

102 

96 

84 

Yes 

E-4 

43 

Male 

12 

252 

42 

42 

Yes 

E-5 

25 

Female 

14 

78 

12 

12 

Yes 

Novices 

N-1 

31 

Male 

16 

0 

0 

0.25 

No 

N-2 

53 

Male 

16 

0 

0 

0 

No 

N-3 

22 

Male 

12 

0 

0 

0 

No 

N-4 

40 

Male 

12 

0 

0 

0 

Yes 

N-5 

24 

Male 

16 

0 

1.5 

1.5 

No 

Table  16.  Breakdown  of  Expert  and  Novice  Demographic  Data. 


Experts 

Novices  | 

□ 

□ 

ta 

n 

□ 

B 

B 

B 

B 

□ 

Age  j 

<  29  years  old 

□ 

□ 

B 

B 

□ 

B 

B 

30-39 

B 

B 

B 

>  40  years  old 

□ 

□ 

B 

□ 

B 

B 

Education  | 

high  school 

B 

B 

□ 

B 

□ 

B 

B 

some  college/assoc,  degree 

B 

B 

bachelor  degree 

□ 

□ 

B 

B 

B 

UXO  Experience  | 

<  6  months 

□ 

B 

B 

B 

B 

B 

6-24  months 

B 

24  -  60  months 

B 

B 

>  60  months 

□ 

B 

B 

□ 

2)  Some  interesting  points  to  notice  from  the  above  tables  are  that  all  the  experts  had  prior 
military  experience  compared  to  only  one  novice.  Also,  only  three  people  had  college  degrees, 
and  they  were  all  novices. 
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3)  Using  the  above  tables,  the  operators’  performance  measurements,  Pd  and  FAR,  with 
and  without  instrumentation  were  plotted  against  age  and  education.  Figures  17-20  display  the 
average  Pd  and  FAR  of  all  operators  over  age  and  education. 


O) 

re 


re 


T3 

Q- 


Probability  of  Detection  vs.  Age  -  Operators  without 
Instrumentation 

UOO  i ^  - 

0.90 - 

0.80 - 

0.70 - 

0.60 - 

0.50 - 

0.40  -| - , - , - 

<  29  30-39  >  40 

Age 


Probability  of  Detection  vs.  Age  -  Operators  with 
Instrumentation 


Age 


Figure  17.  Average  Probability  of  Detection  over  Age  for  All  Operators. 


Figure  18.  Average  False  Alarm  Rate  over  Age  for  All  Operators. 
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Figure  19.  Average  Probability  of  Detection  over  Education  for  All  Operators. 


Figure  20.  Average  False  Alarm  Rate  over  Education  for  All  Operators. 


4)  Figures  21-24  present  the  average  Pd  and  FAR  of  experts  versus  novices  over  age  and 
education.  Figures  21  and  22  indicate  there  is  a  significant  interaction  effect  of  age  between 
experts  and  novices. 
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Figure  2 1 .  Probability  of  Detection  over  Age  for  Experts  vs.  Novices. 
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Probability  of  Detection  vs.  Education  -  Operators  without 
Instrumentation 
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Figure  23.  Probability  of  Detection  over  Education  for  Experts  vs.  Novices. 
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Figure  24.  False  Alarm  Rate  over  Education  for  Experts  vs.  Novices. 
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Appendix  A 
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1 .  Additional  Group  Breakdowns 

a.  Group  1  (High  Performers)  versus  Group  2  (Low  Performers) 

Since  comparing  Experts  and  Novices  resulted  in  most  of  the  novices  performing  better, 
the  consideration  of  rearranging  the  groups  to  find  the  high  and  low  perfonners  and  then 
discovering  what  perfonnance  characteristics  make  them  stand  out  as  “better”  than  the  rest  was 
evolved.  Using  both  the  operators  instrumented  and  uninstrumented  performance  data  (Pd  and 
FAR),  each  operator  was  ranked  in  order  of  best  to  worse  perfonnance  with  and  without 
instrumentation.  Then  those  two  ranks  were  averaged,  and  a  final  ranking  was  made.  The  top 
five  operators  are  called  Group  1  and  the  bottom  five  operators  are  called  Group  2.  Group  1 
consists  of  one  expert  and  four  novices,  and  Group  2  consists  of  four  experts  and  one  novice. 
Table  A-l  presents  a  summary  of  the  performance  data  of  the  two  groups  without 
instrumentation. 


Table  A-l.  Summary  of  Performance  Data  of  Group  1  and  Group  2  without  Instrumentation. 


Operator 

Probability 
of  Detection 
(Pd) 

False 

Alarm 

Rate 

(1/mA2) 

ROC 

Distance 

Lane 

Velocity 

(m/s) 

E-1 

100.0% 

0.154 

0.154 

0.095  | 

N-1 

100.0% 

0.057 

0.057 

0.104  j 

Q_ 

N-2 

98.3% 

0.028 

0.033 

0.112 

3 

o 

N-4 

98.3% 

0.168 

0.169 

0.050  I 

O 

N-5 

93.3% 

0.017 

0.069 

0.094  1 

Mean 

98.0% 

0.085 

0.096 

0.091 

Std.  Dev. 

0.0274 

0.0712 

0.0611 

0.0240 

E-2 

91.7% 

0.072 

0.110 

0.088  [ 

E-3 

100.0% 

0.087 

0.087 

0.100 

CM 

Q_ 

E-4 

93.3% 

0.034 

0.075 

0.092  I 

3 

o 

E-5 

93.3% 

0.090 

0.112 

0.096 

s_ 

(D 

N-3 

95.0% 

0.047 

0.069 

0.124 

Mean 

94.7% 

0.066 

0.091 

0.100 

Std.  Dev. 

0.0321 

0.0250 

0.0199 

0.0142 

Breaking  the  operators  into  the  high  performer  group  and  low  performer  group  resulted  in 
a  larger  the  gap  in  the  mean  Pd  measurement  than  the  expert/novice  breakdown.  However, 
Group  1  has  higher  FAR  and  ROC  Distance.  This  occurred  because  the  breakdown  of  groups 
was  not  solely  based  on  this  data,  but  a  combination  of  this  data  and  the  data  with 
instrumentation.  Using  the  Chi-Square  Distribution  at  0.05  significance  level,  the  Pd  of  Group  1 
was  found  to  be  significantly  greater  than  the  Pd  of  Group  2.  Figure  A-l  illustrates  the  Pd  versus 
FAR  of  Group  1  and  Group  2.  Using  the  Mann-Whitney  Test  at  0.05  significance  level,  no 
significant  differences  were  found  between  the  number  of  false  alarms  and  distance  measurement 
of  Group  1  and  Group  2. 
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Figure  A-l.  Pd  versus  FAR  of  Group  1  and  Group  2  without  Instrumentation. 


Table  A-2  presents  the  ordnance  Pd  of  Group  1  (High  Performers)  vs.  Group  2  (Low 
Perfonners).  Group  1  had  100%  Pd  both  horizontally  and  vertically  with  three  different  types  of 
ordnance:  40mm,  105mm,  and  155mm.  They  also  did  equal  to  or  better  than  the  Group  2  in 
every  category  except  the  60mm  vertical.  Overall,  Group  1  averaged  3.1%  higher  Pd’s  than  the 
Group  2. 


Table  A-2.  Group  1  (High  Performers),  vs.  Group  2  (Low  Performers)  without 
Instrumentation  -  Pd  by  Ordnance  Orientation  and  Type. 


Type 

Group  1 

Group  2 

Total 

Pd  Difference 

40mm 

100.0% 

100.0% 

100.0% 

0.0%  | 

OJ 

+-» 

60mm 

96.0% 

92.0% 

94.0% 

4.0% 

c 

o 

81mm 

86.7% 

73.3% 

80.0% 

13.3% 

N 

z 

105mm 

100.0% 

100.0% 

100.0% 

0.0% 

o 

X 

155mm 

100.0% 

100.0% 

100.0% 

0.0% 

Total 

96.2% 

92.3% 

94.2% 

40mm 

100.0% 

100.0% 

100.0% 

0.0% 

60mm 

97.1% 

100.0% 

98.6% 

-2.9% 

o 

81mm 

97.1% 

88.6% 

92.9% 

8.6% 

t : 

<D 

105mm 

100.0% 

95.0% 

97.5% 

5.0% 

> 

155mm 

100.0% 

97.1% 

98.6% 

2.9% 

Total 

99.4% 

96.5% 

97.9% 

Overall  Total 

98.0% 

94.7% 

96.3% 
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Figures  A-2  -  A-4  show  the  comparison  of  lane  velocity  to  the  three  performance 
measurements,  Pd,  FAR,  and  Dist  ROC,  respectively.  No  trends  are  evident  through  these 
figures. 
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Figure  A-2.  Group  1  and  Group  2  -  Pd  vs.  Lane  Velocity. 
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Figure  A-4.  Group  1  and  Group  2  -  Dist  ROC  vs.  Lane  Velocity. 


b.  Group  1  (High  Performers),  Group  2  (Middle  Performers),  and  Group  3  (Low  Performers) 


Further  exploring  the  idea  of  best  and  worst  performers,  it  seemed  that  there  was  a  clear 
distinction  between  the  top  three  and  bottom  three  performers.  Three  groups  were  then  formed 
based  on  the  previously  discussed  rank  to  detennine  what  characteristics  make  the  high 
performers  better  than  the  rest  and  the  low  performers  worse  than  the  rest.  Group  1  consists  of 
three  novices,  Group  2  contains  two  experts  and  two  novices,  and  Group  3  has  the  remaining 
three  experts.  Table  A-3  presents  a  summary  of  the  performance  data  of  the  three  groups 
without  instrumentation. 


Table  A-3.  Summary  of  Performance  Data  of  Group  1,  Group  2,  and  Group  3  without 

Instrumentation. 


Operator 

Probability 
of  Detection 
(Pd) 

False 

Alarm 

Rate 

(1/mA2) 

ROC 

Distance 

Avg  Lane 
Velocity 
(m/s) 

N-5 

93.3% 

0.017 

0.069 

0.094  | 

Q_ 

N-2 

98.3% 

0.028 

0.033 

0.112 

3 

o 

N-1 

100.0% 

0.057 

0.057 

0.104 

O 

Mean 

97.2% 

0.034 

0.053 

0.103 

Std.  Dev. 

0.028 

0.017 

0.015 

0.007 

N-4 

98.3% 

0.168 

0.169 

0.050  | 

CM 

E-1 

100.0% 

0.154 

0.154 

0.095  1 

Q. 

3 

N-3 

95.0% 

0.047 

0.069 

0.124 

O 

E-3 

100.0% 

0.087 

0.087 

0.100  j 

o 

Mean 

98.3% 

0.114 

0.120 

0.092  | 

Std.  Dev. 

0.020 

0.049 

0.042 

0.027  | 

E-5 

93.3% 

0.090 

0.112 

0.096  j 

CO 

Q_ 

E-2 

91 .7% 

0.072 

0.110 

0.088 

3 

o 

E-4 

93.3% 

0.034 

0.075 

0.092 

i_ 

( D 

Mean 

92.8% 

0.065 

0.099 

0.092 

Std.  Dev. 

0.008 

0.024 

0.017 

0.003 

Breaking  the  operators  into  three  groups  resulted  in  a  larger  the  gap  in  the  mean  Pd  and 
mean  FAR  measurement  between  groups  1  and  3  than  the  expert/novice  breakdown.  To 
statistically  analyze  Pd,  the  Chi-Square  Distribution  at  0.05  significance  level  was  used.  To 
analyze  the  number  of  false  alanns  and  distance  measurement,  the  Mann- Whitney  Test  at  the 
0.05  significance  level  was  used.  For  both  tests,  only  two  groups  can  be  compared  at  a  time. 
Testing  of  Group  1  versus  Group  2  indicated  no  significant  differences  were  found.  Testing  of 
Group  1  versus  Group  3  resulted  in  Group  1  having  significantly  higher  Pd  than  Group  3  and 
also  significantly  shorter  ROC  distance.  Testing  of  Group  2  versus  Group  3  indicated  Group  2 
had  significantly  higher  Pd  than  Group  3.  Despite  several  significant  conclusions,  these  results 
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are  only  for  informational  purposes  because  the  groups  were  purposely  grouped  by  performance. 
Figure  A-5  illustrates  the  Pd  versus  FAR  of  Group  1,  Group  2,  and  Group  3. 


Figure  A-5.  ROC  Plot  of  Group  1  (high  performers),  Group  2  (middle  performers),  and  Group  3 

(low  perfonners)  without  Instrumentation. 


Table  A-4  presents  the  ordnance  Pd  of  Group  1  (High  Performers)  vs.  Group  2  (Middle 
Perfonners)  vs.  Group  3  (Low  Performers).  Groups  1  and  2  had  100%  Pd  both  horizontally  and 
vertically  with  three  different  types  of  ordnance,  40mm,  105mm,  and  155mm.  Group  1  had 
greater  or  equal  Pd  to  Group  3  except  for  the  60mm  vertical.  Group  1  had  trouble  detecting 
60mm  horizontal  and  vertical  and  81mm  horizontal.  Group  3  had  particular  difficulty  with 
8 1mm  horizontal.  Overall,  Group  1  averaged  4.5%  higher  Pd’s  than  the  Group  3. 
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Table  A-4.  Group  1  (High  Performers),  vs.  Group  2  (Middle  Performers)  vs.  Group  3 
(Low  Performers)  without  Instrumentation  -  Pd  by  Ordnance  Type  and  Orientation. 


Type 

Group  1 

Group  2 

Group  3 

Total 

Pd  Difference  - 
Group  1  vs.  Group  3 

40mm 

100.0% 

100.0% 

100.0% 

100.0% 

0.0% 

<0 

*-> 

60mm 

93.3% 

95.0% 

93.3% 

94.0% 

0.0% 

c 

o 

81mm 

83.3% 

91 .7% 

61.1% 

80.0% 

22.2% 

N 

L 

105mm 

100.0% 

100.0% 

100.0% 

100.0% 

0.0% 

o 

X 

155mm 

100.0% 

100.0% 

100.0% 

100.0% 

0.0% 

Total 

94.9% 

97.1% 

89.7% 

94.2% 

1  1 

40mm 

100.0% 

100.0% 

100.0% 

100.0% 

0.0% 

60mm 

95.2% 

100.0% 

100.0% 

98.6% 

-4.8% 

O 

81mm 

100.0% 

92.9% 

85.7% 

92.9% 

14.3% 

o 

105mm 

100.0% 

100.0% 

91.7% 

97.5% 

8.3% 

> 

155mm 

100.0% 

100.0% 

95.2% 

98.6% 

4.8% 

Total 

99.0% 

99.3% 

95.1% 

97.9% 

1  1 

Overall  Total 

97.2% 

98.3% 

92.8% 

96.3% 

Figures  A-5  -  A-7  show  the  comparison  of  lane  velocity  to  the  three  performance 
measurements,  Pd,  FAR,  and  Dist  ROC,  respectively.  The  trend  that  stands  out  the  most  from 
these  plots  is  in  Figure  13,  both  Group  1  and  Group  2  have  lower  Dist  ROC  as  their  Lane 
Velocities  increase. 
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Figure  A-5.  Group  1,  Group  2,  and  Group  3  -  Pd  vs.  Lane  Velocity. 
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Figure  A-7.  Group  1,  Group  2,  and  Group  3  -  Dist  ROC  vs.  Lane  Velocity 
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Figure  A-6.  Group  1,  Group  2,  and  Group  3  -  FAR  vs.  Lane  Velocity 


2.  Ordnance  Type,  Depth,  and  Orientation 


The  greatest  Pd  differences  between  the  groups  occurred  for  the  81mm  horizontal  and 
vertical  and  105mm  vertical.  The  worse  groups  had  trouble  with  this  ordnance,  while  the  better 
groups  excelled.  Table  A-5  provides  the  difference  between  the  Pd  of  the  best  and  worst  group 
for  the  three  classifications. 


Table  A-5.  Greatest  Pd  Differences  per  Group  Breakdowns. 


Group  Breakdown 

81mm  Horizontal 

81mm  Vertical 

105mm  Vertical 

Novices  vs.  Experts 

6.7% 

2.9% 

5.0% 

Group  1  vs.  Group  2 

13.3% 

8.6% 

5.0%  ) 

Group  1  vs.  Group  3 

22.2% 

14.3% 

8.3% 
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APPENDIX  P.  TRACKING  SYSTEM 


P-1 


(Page  P-2  Blank) 


TRACKING  SYSTEM 


Objective 


The  objective  of  this  phase  of  testing  was  to  select  and  utilize  a  tracking  system  capable  of 
recording  human  motion  during  the  UXO  survey,  as  well  as  use  geophysical  data  acquired  on  the 
site  prior  to  testing  as  input  to  a  tracking  system.  This  data  input  enabled  the  system  to  display 
site  boundaries  and  test  item  locations. 

Requirements 

A  system  capable  of  tracking  human  motion  during  a  UXO  survey  was  required  for  this 
test.  The  system  must  allow  for  evaluators  to  review  data  from  each  test  participant  on  a  lane- 
specific  basis.  Additionally,  a  real-time  playback  of  participant’s  motion  was  required.  A  target 
coverage  or  ground  truth  Geographic  Information  System  (GIS)  map  with  site  boundary  points 
was  also  required  for  use  as  input  to  this  tracking  system. 

Test  Procedures 


a.  Identify  human  tracking  system  capable  of  meeting  test  requirements: 

1)  Complete  Commercial  technology  study. 

2)  Communicate  with  Subject  Matter  Experts  within  current  government  mission 

areas. 

3)  Define  and  refine  test  requirements  based  on  a  and  b. 

4)  Evaluate  technologies  based  on  test  requirements. 

5)  Select  best  system  that  meets  criteria. 

b.  Obtain  system. 

1)  Evaluate  current  operational  status. 

2)  Hardware  prove-out. 

3)  Software  prove-out. 

4)  Operational  prove-out. 

c.  Repair,  modify  or  upgrade  based  on  evaluations. 

d.  Train  operators  for  test  scenario. 
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e.  Re-evaluate  the  system  for  pre-testing. 

f.  Determine  if  system  will  meet  test  specifications. 

Data  Required 

The  system  must  provide  evaluators  with  detector  sweep  height  and  sweep  rate. 
Additionally,  it  must  record  and  verify  the  dynamic  motion  of  test  participant  as  they  traverse  the 
test  grid.  In  general,  data  required  must  fulfill  test  requirements  described  in  the  Operator 
Influence  on  Sensor  Technologies  Detailed  Test  Plan. 

All  emplaced  items  and  boundary  points  within  the  test  bed  were  recorded  in  the  form  of 
NAD83  Northing/Easting  coordinate  locations. 

Outcomes 


System  Identification 

The  first  step  in  the  process  was  to  locate  and  identify  a  system  that  satisfied  test  criteria. 
A  complete  commercial  market  research  study  was  initiated  for  a  previous  test  effort  by 
members  of  the  ATC  Geodetics  team.  Factors  such  as  usability,  maintainability,  technology 
maturity,  and  adaptability  to  various  testing  scenarios  were  reviewed  during  this  process.  The 
results  of  this  study  detennined  that  the  Threat  Management  System  (TMS)  was  a  suitable  and 
rugged  platform  for  human  motion  tracking  in  an  urban  environment.  Although,  designed  for 
minefield  training  scenarios,  it  was  assumed  that  this  system  would  be  suitable  for  the  purposes 
of  this  test  based  on  clear  similarities  in  de-mining  and  UXO  sweep  operations.  The  system 
provided  a  comprehensive  platform  that  would  allow  most  test  requirements  to  be  met. 

Threat  Management  System  Introduction 

a.  The  TMS  system  consisted  of  a  laser  based  motion  tracking  system  linked  directly  to  a 
telemetry  data  acquisition  system.  A  PC-based  Field  Instrumentation  Unit  (FIU)  translated  three 
dimensional  coordinates  to  a  secondary  PC-based  system  that  recorded  real  time  positioning. 
The  two  systems  communicated  via  wireless  gateway  based  on  IEEE  802.11b  interface 
requirements.  A  TMS  operator  monitored  the  telemetry  link  to  ensure  that  data  was  flowing 
from  the  laser  sensors  through  the  communication’s  link.  A  detailed  overview  of  this  system  can 
be  found  in  the  TMS  Systems  User  manual  in  appendix  L. 

b.  The  TMS  platform  was  originally  engineered  for  the  Program  Office  for  Simulations, 
Training  and  Instrumentation  (PEO  STRI).  The  US  Army  Threat  Systems  Management  Office 
(TSMO)  program  was  the  main  authority  upon  the  platform’s  inception  during  FY2001.  The 
TMS  system  facilitates  de-mining  testing  and  training  in  either  real  or  virtual  environments.  The 
system  provides  virtual  mines  for  inclusion  into  exercises  (TMS  Systems  Manual). 
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Obtaining  the  System 


a.  After  contacting  members  of  PEO  STRI,  ATC  Geodetics  team  members  found  that  the 
technology  required  for  tracking  human  motion  was  available,  however  the  TMS  hardware  and 
associated  components  were  mothballed  due  to  budget  constraints.  PEO  STRI  authorized  ATC 
to  utilize  the  system  for  testing  requirements.  ATC  Geodetics  personnel  acquired  the  TMS 
hardware,  trained  on  associated  software  and  attempted  to  set-up  the  complete  system  onsite  at 
Aberdeen  Test  Center. 

b.  The  complete  TMS  was  then  tested  for  operational  utility  at  ATC’s  Standardized  UXO 
Site.  This  process  entailed  three  main  procedures;  hardware  configuration,  software 
configuration,  and  operational  prove-out.  Hardware  configuration  was  defined  as  obtaining  all 
necessary  components  and  computer  interfaces  required  to  obtain  operational  status.  Software 
configuration  included  obtaining  necessary  software,  troubleshooting  interfaces  between 
hardware,  and  verifying  data  reporting  in  required  fonnats.  Operational  prove-out  involved 
pre-testing  onsite,  tracking  operators  traverse  a  scale  plot  of  land  similar  to  those  found  on  the 
Standardized  UXO  Test  Site. 

c.  Geophysical  referencing  data  was  collected  on-site  using  a  Trimble  GPS.  The  locations 
of  all  targets  and  boundaries  were  recorded  and  mapped.  Target  properties  known  to  influence 
detection  (type,  ferrous/nonferrous,  depth,  and  orientation)  were  annotated  for  each.  The 
compiled  data  was  used  as  input  to  the  TMS,  prior  to  test  commencement,  in  the  form  of  the  test 
bed  map.  The  data  input  allowed  the  TMS  to  provide  an  accurate  and  visual  readout  of  operator 
location  with  respect  to  the  test  items.  Input  of  the  site  boundary  points  made  available  clear 
delineation  of  the  test  bed  borders.  Geodetics  personnel  documented  the  data,  on-site,  when 
emplacement  was  completed. 

System  prove-out 

a.  The  TMS  platform  provided  a  fundamental  design  that  contained  high  potential  to 
satisfy  test  requirements.  The  TMS  project  effort  had  been  stopped  one  year  prior  to  ATC 
contacting  PEO  -  SRI  to  inquire  on  utilizing  the  system  for  the  Operator  Influence  on  UXO 
Sensor  testing.  Thus,  the  hardware  was  in  a  warehouse-containerized  state  when  ATC  personnel 
received  the  complete  unit.  After  piecing  the  system  together  and  perfonning  system  checks,  a 
preliminary  prove-out  test  was  completed. 

b.  On  the  basis  of  mobility,  the  TMS  system  was  placed  in  an  ATC  data  van  and  powered 
from  an  external  power  source.  If  a  regulated  hard  line  electrical  source  was  available,  the 
system  was  powered  by  a  standard  shore  power  120V  AC  connection.  When  deployed  in  field 
conditions,  the  system  was  powered  by  a  model  806A  60kW  Tactically  Quiet  Generator.  The 
generator  was  fueled  with  DF2  diesel  fuel.  Daily  Inspections  were  made  to  the  power  generation 
system,  to  include  connections  and  wiring  to  the  data  van. 
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c.  The  Hardware  Configuration  process  yielded  many  noticeable  issues  with  the  TMS.  Of 
the  two  operator  stations  both  experienced  intermittent  operational  status  during  the  course  of 
prove  out.  It  was  determined  that  this  was  caused  by  fluctuations  from  the  60kW  TCQ  set 
coupled  with  low  battery  back-up  systems.  Additionally,  loose  wiring  and  connections  were 
observed  throughout  the  setup  during  short  movements  of  the  data  van.  Ancillary  cards 
contained  onboard  both  operator  stations  (PC  motherboards)  were  damaged  and  required 
replacement.  These  consisted  of  video  and  network  communications  cards. 

d.  The  TMS  utilized  four  rotating  laser  light  energy  sources  that  were  positioned  on  the 
perimeter  of  the  test  grid.  The  radial  effectiveness  of  each  transmitter  was  found  to  be 
approximately  250  degrees  with  a  maximum  linear  range  of  130  meters.  Ranges  were  observed 
to  be  highly  dependent  on  environmental  conditions  such  as  ambient  light,  wind-speed  and 
temperature.  The  four  individual  laser  emitters  were  powered  by  eight  Lithium  Ion  D  Cell 
rechargeable  batteries.  Operating  times  between  charges  ranged  from  30  minutes  to  3  hours.  It 
was  determined  that  because  of  the  time  that  batteries  were  in  storage  prior  to  set-up  and  the 
tendency  for  Li  ion  batteries  to  deteriorate  in  such  conditions,  the  majority  of  the  batteries 
supplied  with  the  TMS  were  deemed  unusable  for  the  test.  It  was  observed,  however,  that 
batteries  that  were  cycled  on  a  regular  basis  provided  greater  time  between  charges. 

e.  The  laser  tracking  would  self-calibrate,  however  a  notable  signal  drift  was  observed 
over  an  extended  time  domain.  It  was  noted  that  the  first  two  hours  of  continuous  tracking 
indicated  under  2  cm  signal  drift,  however  time  periods  of  over  two  hours  yielded  exponentially 
higher  signal  drift.  Accuracy  low,  however  results  location  error  was  consistent  and  repeatable. 

f.  Laser  receivers  located  on  each  participant’s  foot  as  well  as  the  shaft  of  the  UXO 
sensor  provided  absolute  coordinates  to  a  data  acquisition  system  via  wireless  telemetry  link.  It 
was  observed  that  the  wiring  and  connections  of  the  foot  and  shaft  mounted  sensors  were  very 
sensitive  to  shock  and  vibration. 

g.  The  laser  sensors  were  hardwired  to  the  PC  based  Field  Instrumentation  Unit  (FIU), 
which  was  a  self-contained  communication  link  and  position  calculation  device  powered  by  a 
SINGARS  style  rechargeable  lithium  ion  30V  5.5  amp-hour  battery.  The  FIU  was  linked  to  the 
Operator  stations,  which  recorded  the  conditioned  signals  from  the  laser  sensors.  The  FIU  was 
designed  to  be  worn  around  the  waist  of  the  operator,  allowing  easy  connectivity  to  the  laser 
sensors  and  geophysical  detection  device. 

h.  It  was  observed  that  the  complete  FIU  with  battery  was  cumbersome  and  induced 
stressors  on  individuals  while  traversing  the  test  plot.  Alternative  strategies  consisted  of 
allowing  a  second  individual  to  carry  the  FIU  with  battery  and  towing  the  FIU  behind  the 
participant  in  a  hand-cart.  The  first  alternative  proved  to  be  difficult  as  the  connecting  wires 
between  the  laser  sensors  and  the  FIU  were  designed  for  very  short  lengths,  thus  the  second 
individual  was  required  to  walk  in  close  proximity  to  the  test  participant.  Due  to  the  sometimes 
erratic  motions  of  the  test  participants,  the  second  individual  would  make  contact  and  distract  the 
test  participant.  This  would  not  provide  an  ideal  situation  required  for  test  requirements.  The 
second  method,  which  involved  towing  the  FIU  behind  the  test  participant  was  successful, 
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however,  required  an  extension  of  the  cabling  connecting  the  laser  sensors  and  FIU.  The  cabling 
proved  difficult  to  modify,  as  unique  connectors  to  the  FIU  were  used  in  the  original  design. 


i.  A  video  feed  channel  was  available;  however  no  camera  or  video  feed  path  was 
installed  on  initial  set-up.  A  USB  thumb-style  camera  was  plugged  directly  into  the  operator  unit 
and  video  feed  was  found  to  be  operational,  however  the  feed’s  reliability  was  intermittent. 
Troubleshooting  revealed  that  this  was  caused  by  either  poor  connection  within  the  FIU,  or 
insufficient  bandwidth  between  the  FIU  and  the  Operator  stations.  Thus,  the  IEEE  802.11b 
interface  seemed  to  inadequate  for  the  video  feed  and  position  calculations  to  be  sent 
simultaneously  between  the  FIU  and  operator  station.  The  video  feed  was  unreliable  in 
operation. 

j.  The  operational  prove-out  of  the  system  provided  a  rapid  fielding  situation  that  was 
intended  to  allow  data  collected  during  prove-out  to  be  utilized  in  addition  to  data  collected 
during  full  scale  test  work.  The  system  tracked  human  movement  32%  of  the  time  that  was 
required.  Additionally,  signal  dropout  various  hardware  failures  and  system  power  requirements 
were  concerns  that  required  attention. 

Upgrading  the  TMS 

a.  The  integrated  test  team  determined  that  upgrades  to  the  existing  TMS  would  be 
required  in  order  to  meet  test  schedule  requirements  based  on  the  results  of  the  operational 
prove-out. 

b.  Members  contacted  the  developer  of  the  TMS;  Scientific  Research  Corporation,  Inc. 
The  results  of  the  TMS  system  prove-out  were  relayed  to  the  developers  at  SRC.  Discussion  was 
initiated  concerning  possible  improvements  to  the  existing  TMS  for  use  under  the  Operator 
Influence  Test  plan.  SRC  provided  an  overview  of  options  that  would  allow  for  improvement  of 
the  existing  system.  The  requirements  from  the  prove-out  and  SRC’s  recommendations  can  be 
found  in  appendix  M.  A  Firm  fixed  cost  contract  was  initiated  with  SRC  and  funded  with  project 
funds. 


c.  The  complete  system,  consisting  of  all  hardware  and  software,  was  delivered  to  SRC 
via  data  van  from  ATC.  At  this  time,  developers  were  able  to  first-hand  troubleshoot,  diagnose, 
repair  and  modify  as  necessary  using  a  systems  engineering  strategy. 

d.  The  two  operator  stations  within  the  data  van  were  removed,  inspected  and  made 
functional  by  replacing  hardware  components  within  the  computers.  Loose  cables  were 
identified  and  replacement  components  were  replaced  or  procured.  The  802.11b  wireless  card 
was  upgraded  to  an  802. 1  lg  protocol.  This  was  tested  and  increased  reliability,  but  also 
communication  speed  to  accommodate  a  new  USB  camera. 

e.  The  individual  FIU’s  were  observed  to  be  creating  large  amounts  of  heat  from  the  CPU 
contained  in  the  airtight  container.  Temperatures  were  estimated  to  be  above  45  degrees 
centigrade.  There  was  concern  that  the  high  temperatures  might  damage  the  FIU  circuitry  within 
the  airtight  container.  A  heat  sink  was  designed  and  fabricated  to  attach  directly  to  the  large 
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surface  area  of  the  FIU.  Additionally,  two  small  fans  were  placed  on  opposing  sides  of  the  FIU 
container  to  allow  for  cooling  of  internal  components. 

f.  The  Arc  Second  laser  transmitters  were  functional,  however  the  batteries  that  provided 
power  were  observed  to  have  low  endurance  limits.  New  batteries  and  complete  battery  trays 
were  ordered. 

g.  These  improvements  were  made  onsite  at  the  SRC  facilities  in  Huntsville,  AL. 

h.  The  system  was  then  packaged  for  shipment  and  sent  to  Aberdeen  Test  Center.  The 
complete  system  was  setup  at  the  Standardized  UXO  test  site  to  simulate  testing  conditions. 
Members  of  the  SRC  test  staff  were  on-hand  to  complete  on-site  diagnosis  and  repairs  of  any 
travel-related  complications  found. 

i.  At  this  time,  it  was  determined  that  the  network  interface  (NIC)  cards  on  each  operator 
station  were  damaged  and  inoperable  as  well  as  the  battery  back-up  system.  Both  were  replaced 
onsite  by  SRC  technicians. 

j.  Upon  completion  of  set-up  at  ATC,  the  system  was  fully  functional  with  improved 
reliability,  endurance  and  range. 

k.  The  data  collected  as  a  result  of  the  geophysical  survey  was  downloaded  using 
computer-assisted  design  (CAD)  and  GIS  software  platfonns.  The  UTM  coordinates  were 
maintained  by  the  ATC  test  director  and  ATC  geodetics  staff. 
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A&M 

ACE 

ADST 

APG 

ARL 

ATC 

ATSS 

BAR 

BARres 

CAD 

CFA 

CTC 

DOD 

DTC 

DTP 

EEE 

EOD 

EPA 

EQT 

FA 

FAR 

fpres 

GIS 

GPS 

IUTP 

LED 

Log. Sec 

METDC 

Pd 

Pfa 

Pip"8 

P/" 

PTA 

R&D 

RTK 

TAMUS 

TEEX 

TMS 

USAEC 

UTM 

UXO 


Agricultural  and  Mechanical 

U.S.  Army  Corps  of  Engineers 

Aberdeen  Data  Services  Team 

Aberdeen  Proving  Ground 

U.S.  Army  Research  Laboratory 

U.S.  Army  Aberdeen  Test  Center 

Aberdeen  Test  and  Support  Services 

background  alarm  rate 

response  stage  background  alarm  rate 

computer-assisted  design 

common  false  alarms 

Concurrent  Technologies  Corporation 

Department  of  Defense 

U.S.  Army  Developmental  Test  Command 

Detailed  Test  Plan 

explosives  and  explosive  effects 

Explosive  Ordnance  Disposal 

Environmental  Protection  Agency 

Environmental  Quality  Technology 

false  alarm 

false  alann  rate 

response  stage  false  positive 

Geographic  Information  System 

Global  Positioning  System 

International  Unexploded  Ordnance  Training  Program 
light  emitting  diode 

Logistics  Engineering  and  Infonnation  Technology  Company 

Military  Environmental  Technology  Demonstration  Center 

probability  of  detection 

probability  of  false  alann  rates 

response  stage  probability  of  false  positive 

response  stage  probability  of  detection 

pure  tonal  average 

research  and  development 

real-time  kinematic 

Texas  Agricultural  and  Mechanical  University  System 

Texas  Engineering  Extension  Service 

Threat  Management  System 

U.S.  Army  Environmental  Command 

Universal  Transverse  Mercator 

unexploded  ordnance 
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